[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:21:02 dist-test-slave-414w krb5kdc[4339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:21:02 dist-test-slave-414w krb5kdc[4339](info): set up 2 sockets
Jun 20 09:21:02 dist-test-slave-414w krb5kdc[4339](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260620 09:21:03.129662 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:21:03 dist-test-slave-414w krb5kdc[4339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:21:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:21:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:21:03.248817 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:38059
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:38059
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:03.329555 4352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:03.329761 4352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:03.329783 4352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:03.330971 4352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:03.331015 4352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:03.331030 4352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:03.331043 4352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:03.331055 4352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:21:03.332520 4352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:38059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:38059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:03.332774 4352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:03.333106 4352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:03.338171 4358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:03.338215 4360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:03.338200 4357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:03.338585 4352 server_base.cc:1034] running on GCE node
I20260620 09:21:03.338838 4352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:03.339159 4352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:03.340333 4352 hybrid_clock.cc:648] HybridClock initialized: now 1781947263340310 us; error 42 us; skew 500 ppm
Jun 20 09:21:03 dist-test-slave-414w krb5kdc[4339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947263, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:03.342883 4352 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:21:03.343331 4352 webserver.cc:460] Webserver started at http://127.4.59.62:41513/ using document root <none> and password file <none>
I20260620 09:21:03.343550 4352 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:03.343596 4352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:03.343726 4352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:03.344761 4352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "c04fb913f7a84316b3e453c3112d0460"
format_stamp: "Formatted at 2026-06-20 09:21:03 on dist-test-slave-414w"
server_key: "b0558c0486b87ac28bd2653af5a978ce"
server_key_iv: "0cf003269b58ee752962e224a09b6889"
server_key_version: "encryptionkey@0"
I20260620 09:21:03.345124 4352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "c04fb913f7a84316b3e453c3112d0460"
format_stamp: "Formatted at 2026-06-20 09:21:03 on dist-test-slave-414w"
server_key: "b0558c0486b87ac28bd2653af5a978ce"
server_key_iv: "0cf003269b58ee752962e224a09b6889"
server_key_version: "encryptionkey@0"
I20260620 09:21:03.346500 4352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:03.347229 4367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:03.347400 4352 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:03.347499 4352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "c04fb913f7a84316b3e453c3112d0460"
format_stamp: "Formatted at 2026-06-20 09:21:03 on dist-test-slave-414w"
server_key: "b0558c0486b87ac28bd2653af5a978ce"
server_key_iv: "0cf003269b58ee752962e224a09b6889"
server_key_version: "encryptionkey@0"
I20260620 09:21:03.347580 4352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:03.374819 4352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:03.377261 4352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:03.377432 4352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:03.381502 4352 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:38059
I20260620 09:21:03.381541 4419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:38059 every 8 connection(s)
I20260620 09:21:03.381899 4352 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:21:03.382905 4420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:03.383606 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 4352
I20260620 09:21:03.383709 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:21:03.384496 4332 external_mini_cluster.cc:1428] Setting key 9a7fa62eac9250e8a1f84f10df8352e4
I20260620 09:21:03.386108 4420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460: Bootstrap starting.
I20260620 09:21:03.386799 4420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:03.387125 4420 log.cc:824] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:03.387878 4420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460: No bootstrap required, opened a new log
I20260620 09:21:03.389688 4420 raft_consensus.cc:348] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c04fb913f7a84316b3e453c3112d0460" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 38059 } }
I20260620 09:21:03.389840 4420 raft_consensus.cc:374] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:03.389879 4420 raft_consensus.cc:729] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c04fb913f7a84316b3e453c3112d0460, State: Initialized, Role: FOLLOWER
I20260620 09:21:03.390002 4420 consensus_queue.cc:260] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c04fb913f7a84316b3e453c3112d0460" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 38059 } }
I20260620 09:21:03.390086 4420 raft_consensus.cc:388] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:21:03.390120 4420 raft_consensus.cc:482] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:21:03.390174 4420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:03.391181 4420 raft_consensus.cc:504] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c04fb913f7a84316b3e453c3112d0460" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 38059 } }
I20260620 09:21:03.391292 4420 leader_election.cc:302] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c04fb913f7a84316b3e453c3112d0460; no voters:
I20260620 09:21:03.391533 4420 leader_election.cc:288] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:21:03.391618 4425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:03.391759 4425 raft_consensus.cc:686] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [term 1 LEADER]: Becoming Leader. State: Replica: c04fb913f7a84316b3e453c3112d0460, State: Running, Role: LEADER
I20260620 09:21:03.391788 4420 sys_catalog.cc:564] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:03.391861 4425 consensus_queue.cc:237] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c04fb913f7a84316b3e453c3112d0460" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 38059 } }
I20260620 09:21:03.392184 4426 sys_catalog.cc:455] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c04fb913f7a84316b3e453c3112d0460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c04fb913f7a84316b3e453c3112d0460" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 38059 } } }
I20260620 09:21:03.392309 4426 sys_catalog.cc:458] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:03.392524 4427 sys_catalog.cc:455] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c04fb913f7a84316b3e453c3112d0460. Latest consensus state: current_term: 1 leader_uuid: "c04fb913f7a84316b3e453c3112d0460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c04fb913f7a84316b3e453c3112d0460" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 38059 } } }
I20260620 09:21:03.392585 4427 sys_catalog.cc:458] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:03.392702 4434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:21:03.393254 4434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:21:03.395161 4434 catalog_manager.cc:1347] Generated new cluster ID: 50213270dc534ffc82c3dc6260c28ea9
I20260620 09:21:03.395216 4434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 09:21:03 dist-test-slave-414w krb5kdc[4339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:03.402879 4422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:03.385459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53344 (local address 127.4.59.62:38059)
0620 09:21:03.385648 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:03.385656 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:03.386004 (+ 348us) server_negotiation.cc:407] Connection header received
0620 09:21:03.389678 (+ 3674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:03.389684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:03.389732 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:03.389867 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:03.392050 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.393545 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.394883 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.395425 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.395561 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:03.398834 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:03.398837 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:03.398838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:03.398847 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:03.400964 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:03.401470 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:03.401473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:03.401473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:03.401595 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:03.401852 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:03.401853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:03.401853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:03.402090 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:03.402265 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:03.402516 (+ 251us) server_negotiation.cc:299] Negotiation successful
0620 09:21:03.402651 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260620 09:21:03.403314 4434 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:21:03.403894 4434 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:21:03.409823 4434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c04fb913f7a84316b3e453c3112d0460: Generated new TSK 0
I20260620 09:21:03.410002 4434 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:21:03.436586 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:38059
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:03.518816 4446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:03.519017 4446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:03.519042 4446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:03.520179 4446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:03.520219 4446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:03.520246 4446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:21:03.521689 4446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:38059
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:03.521914 4446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:03.522166 4446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:03.527004 4452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:03.527068 4454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:03.527300 4446 server_base.cc:1034] running on GCE node
W20260620 09:21:03.527077 4451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:03.527525 4446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:03.527732 4446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:03.528921 4446 hybrid_clock.cc:648] HybridClock initialized: now 1781947263528911 us; error 29 us; skew 500 ppm
Jun 20 09:21:03 dist-test-slave-414w krb5kdc[4339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947263, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:03.531638 4446 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:21:03.532088 4446 webserver.cc:460] Webserver started at http://127.4.59.1:39379/ using document root <none> and password file <none>
I20260620 09:21:03.532285 4446 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:03.532325 4446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:03.532415 4446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:03.533527 4446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "b0c1dcbe3fad4110ac1625867bdbc29e"
format_stamp: "Formatted at 2026-06-20 09:21:03 on dist-test-slave-414w"
server_key: "0b04dd8fc82fb67f3bcfbc9aa145ef2f"
server_key_iv: "3c2309202adc0dc13190fc6528f6a519"
server_key_version: "encryptionkey@0"
I20260620 09:21:03.533838 4446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "b0c1dcbe3fad4110ac1625867bdbc29e"
format_stamp: "Formatted at 2026-06-20 09:21:03 on dist-test-slave-414w"
server_key: "0b04dd8fc82fb67f3bcfbc9aa145ef2f"
server_key_iv: "3c2309202adc0dc13190fc6528f6a519"
server_key_version: "encryptionkey@0"
I20260620 09:21:03.535286 4446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:21:03.535950 4461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:03.536144 4446 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:03.536234 4446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "b0c1dcbe3fad4110ac1625867bdbc29e"
format_stamp: "Formatted at 2026-06-20 09:21:03 on dist-test-slave-414w"
server_key: "0b04dd8fc82fb67f3bcfbc9aa145ef2f"
server_key_iv: "3c2309202adc0dc13190fc6528f6a519"
server_key_version: "encryptionkey@0"
I20260620 09:21:03.536315 4446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:03.564093 4446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:03.566715 4446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:03.566885 4446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:03.567145 4446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:03.567456 4446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:03.567509 4446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:03.567551 4446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:03.567579 4446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:03.573499 4446 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:37009
I20260620 09:21:03.573544 4574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:37009 every 8 connection(s)
I20260620 09:21:03.573870 4446 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
Jun 20 09:21:03 dist-test-slave-414w krb5kdc[4339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947263, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:03.581349 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 4446
I20260620 09:21:03.581444 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:21:03.581626 4332 external_mini_cluster.cc:1428] Setting key 212ef7a5e2059c5511e596b08b6fc505
I20260620 09:21:03.583824 4422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:03.574820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:49083 (local address 127.4.59.62:38059)
0620 09:21:03.574872 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:03.574874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:03.575047 (+ 173us) server_negotiation.cc:407] Connection header received
0620 09:21:03.575567 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:03.575568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:03.575583 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:03.575653 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:03.576618 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.577208 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.578215 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.578442 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.578528 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:03.580652 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:03.580653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:03.580653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:03.580663 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:03.582403 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:03.582824 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:03.582825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:03.582826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:03.582889 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:03.583107 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:03.583108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:03.583108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:03.583251 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:03.583318 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:03.583592 (+ 274us) server_negotiation.cc:299] Negotiation successful
0620 09:21:03.583692 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 09:21:03.584007 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:03.574749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:38059 (local address 127.4.59.1:49083)
0620 09:21:03.574900 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:03.574943 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:03.575458 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:03.575744 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:03.575748 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:03.576316 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:03.576512 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:03.576516 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.577299 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.577300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:03.578101 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:03.578103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.578537 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.578539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:03.578667 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:03.579213 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:03.579221 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:03.580548 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:03.582503 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:03.582509 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:03.582511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:03.582744 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:03.582957 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:03.582958 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:03.582958 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:03.583054 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:03.583316 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:03.583321 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:03.583436 (+ 115us) client_negotiation.cc:769] Sending connection context
0620 09:21:03.583553 (+ 117us) client_negotiation.cc:240] Negotiation successful
0620 09:21:03.583743 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260620 09:21:03.584719 4575 heartbeater.cc:347] Connected to a master server at 127.4.59.62:38059
I20260620 09:21:03.584821 4575 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:03.585021 4575 heartbeater.cc:511] Master 127.4.59.62:38059 requested a full tablet report, sending...
I20260620 09:21:03.585573 4384 ts_manager.cc:194] Registered new tserver with Master: b0c1dcbe3fad4110ac1625867bdbc29e (127.4.59.1:37009)
I20260620 09:21:03.586498 4384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:49083
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:21:03.609768 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:38059
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:03.689605 4580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:03.689786 4580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:03.689808 4580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:03.690876 4580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:03.690909 4580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:03.690935 4580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:21:03.692351 4580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:38059
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:03.692576 4580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:03.692811 4580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 09:21:03.697871 4580 server_base.cc:1034] running on GCE node
W20260620 09:21:03.697852 4588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:03.697842 4586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:03.697855 4585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:03.698343 4580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:03.698603 4580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:03.699751 4580 hybrid_clock.cc:648] HybridClock initialized: now 1781947263699739 us; error 29 us; skew 500 ppm
Jun 20 09:21:03 dist-test-slave-414w krb5kdc[4339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947263, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:03.702299 4580 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:21:03.702744 4580 webserver.cc:460] Webserver started at http://127.4.59.2:39007/ using document root <none> and password file <none>
I20260620 09:21:03.702932 4580 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:03.702972 4580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:03.703066 4580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:03.704061 4580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "dec7b7e6694b4be2b1e84c4208d7f4cc"
format_stamp: "Formatted at 2026-06-20 09:21:03 on dist-test-slave-414w"
server_key: "9bd227e941cfc60bdfb398c9f316bcf9"
server_key_iv: "1004dcc4a7d3267c545b440ac4bd55ad"
server_key_version: "encryptionkey@0"
I20260620 09:21:03.704371 4580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "dec7b7e6694b4be2b1e84c4208d7f4cc"
format_stamp: "Formatted at 2026-06-20 09:21:03 on dist-test-slave-414w"
server_key: "9bd227e941cfc60bdfb398c9f316bcf9"
server_key_iv: "1004dcc4a7d3267c545b440ac4bd55ad"
server_key_version: "encryptionkey@0"
I20260620 09:21:03.705643 4580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:21:03.706354 4595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:03.706558 4580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:21:03.706633 4580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "dec7b7e6694b4be2b1e84c4208d7f4cc"
format_stamp: "Formatted at 2026-06-20 09:21:03 on dist-test-slave-414w"
server_key: "9bd227e941cfc60bdfb398c9f316bcf9"
server_key_iv: "1004dcc4a7d3267c545b440ac4bd55ad"
server_key_version: "encryptionkey@0"
I20260620 09:21:03.706722 4580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:03.725358 4580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:03.727242 4580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:03.727380 4580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:03.727619 4580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:03.727917 4580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:03.727972 4580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:03.728012 4580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:03.728050 4580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:03.733974 4580 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:39083
I20260620 09:21:03.734022 4708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:39083 every 8 connection(s)
I20260620 09:21:03.734318 4580 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:21:03.734530 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 4580
I20260620 09:21:03.734835 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:21:03.735036 4332 external_mini_cluster.cc:1428] Setting key b1f80dc36be5ec21f599b2e3d93c96d3
Jun 20 09:21:03 dist-test-slave-414w krb5kdc[4339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947263, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
I20260620 09:21:03.744274 4422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:03.735161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:58779 (local address 127.4.59.62:38059)
0620 09:21:03.735234 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:03.735236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:03.735487 (+ 251us) server_negotiation.cc:407] Connection header received
0620 09:21:03.736048 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:03.736049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:03.736061 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:03.736116 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:03.737039 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.737602 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.738617 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.738817 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.738896 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:03.741458 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:03.741459 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:03.741460 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:03.741467 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:03.742872 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:03.743373 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:03.743374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:03.743375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:03.743427 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:03.743636 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:03.743637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:03.743637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:03.743738 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:03.743789 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:03.744125 (+ 336us) server_negotiation.cc:299] Negotiation successful
0620 09:21:03.744208 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 09:21:03.744341 4711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:03.735173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:38059 (local address 127.4.59.2:58779)
0620 09:21:03.735353 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:03.735398 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:03.735950 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:03.736185 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:03.736189 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:03.736739 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:03.736923 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:03.736927 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.737697 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.737698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:03.738497 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:03.738499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.738935 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.738937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:03.739059 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:03.739599 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:03.739608 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:03.741348 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:03.742990 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:03.742996 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:03.742999 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:03.743300 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:03.743488 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:03.743489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:03.743489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:03.743581 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:03.743793 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:03.743796 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:03.743937 (+ 141us) client_negotiation.cc:769] Sending connection context
0620 09:21:03.744039 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 09:21:03.744180 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260620 09:21:03.744814 4709 heartbeater.cc:347] Connected to a master server at 127.4.59.62:38059
I20260620 09:21:03.744936 4709 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:03.745173 4709 heartbeater.cc:511] Master 127.4.59.62:38059 requested a full tablet report, sending...
I20260620 09:21:03.745594 4384 ts_manager.cc:194] Registered new tserver with Master: dec7b7e6694b4be2b1e84c4208d7f4cc (127.4.59.2:39083)
I20260620 09:21:03.745929 4384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:58779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:21:03.760438 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:38059
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:03.849218 4714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:03.849419 4714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:03.849443 4714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:03.850764 4714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:03.850811 4714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:03.850839 4714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:21:03.852300 4714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:38059
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:03.852552 4714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:03.852797 4714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:03.857960 4722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:03.858119 4714 server_base.cc:1034] running on GCE node
W20260620 09:21:03.858140 4720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:03.858050 4719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:03.858453 4714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:03.858696 4714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:03.859865 4714 hybrid_clock.cc:648] HybridClock initialized: now 1781947263859853 us; error 27 us; skew 500 ppm
Jun 20 09:21:03 dist-test-slave-414w krb5kdc[4339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947263, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:03.862351 4714 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:21:03.862813 4714 webserver.cc:460] Webserver started at http://127.4.59.3:38249/ using document root <none> and password file <none>
I20260620 09:21:03.863049 4714 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:03.863092 4714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:03.863229 4714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:03.864254 4714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "e95e6893457641b2baeae824455ee646"
format_stamp: "Formatted at 2026-06-20 09:21:03 on dist-test-slave-414w"
server_key: "972cb9a9a94a3c3881cc053eef89c849"
server_key_iv: "92d15595b0651753705d220c57ae90cd"
server_key_version: "encryptionkey@0"
I20260620 09:21:03.864609 4714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "e95e6893457641b2baeae824455ee646"
format_stamp: "Formatted at 2026-06-20 09:21:03 on dist-test-slave-414w"
server_key: "972cb9a9a94a3c3881cc053eef89c849"
server_key_iv: "92d15595b0651753705d220c57ae90cd"
server_key_version: "encryptionkey@0"
I20260620 09:21:03.866005 4714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:21:03.866751 4729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:03.866941 4714 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:21:03.867023 4714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "e95e6893457641b2baeae824455ee646"
format_stamp: "Formatted at 2026-06-20 09:21:03 on dist-test-slave-414w"
server_key: "972cb9a9a94a3c3881cc053eef89c849"
server_key_iv: "92d15595b0651753705d220c57ae90cd"
server_key_version: "encryptionkey@0"
I20260620 09:21:03.867079 4714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:03.882578 4714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:03.884505 4714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:03.884613 4714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:03.884788 4714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:03.885012 4714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:03.885041 4714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:03.885063 4714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:03.885075 4714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:03.890722 4714 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:33379
I20260620 09:21:03.890790 4842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:33379 every 8 connection(s)
I20260620 09:21:03.891084 4714 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:21:03.895244 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 4714
I20260620 09:21:03.895349 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:21:03.895511 4332 external_mini_cluster.cc:1428] Setting key bd06938383601612abe62f14c5a3e263
Jun 20 09:21:03 dist-test-slave-414w krb5kdc[4339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947263, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:03.900079 4422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:03.891731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:58555 (local address 127.4.59.62:38059)
0620 09:21:03.891771 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:03.891773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:03.892016 (+ 243us) server_negotiation.cc:407] Connection header received
0620 09:21:03.892612 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:03.892613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:03.892624 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:03.892681 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:03.893665 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.894144 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.895217 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.895477 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.895570 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:03.897786 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:03.897787 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:03.897788 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:03.897793 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:03.898927 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:03.899303 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:03.899304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:03.899305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:03.899363 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:03.899574 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:03.899575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:03.899575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:03.899680 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:03.899732 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:03.899952 (+ 220us) server_negotiation.cc:299] Negotiation successful
0620 09:21:03.900015 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 09:21:03.900300 4845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:03.891719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:38059 (local address 127.4.59.3:58555)
0620 09:21:03.891896 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:03.891938 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:03.892417 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:03.892746 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:03.892749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:03.893390 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:03.893587 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:03.893592 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.894293 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.894294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:03.895077 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:03.895079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.895578 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.895579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:03.895728 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:03.896288 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:03.896298 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:03.897707 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:03.898995 (+ 1288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:03.899000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:03.899002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:03.899224 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:03.899429 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:03.899430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:03.899430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:03.899520 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:03.899735 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:03.899738 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:03.899848 (+ 110us) client_negotiation.cc:769] Sending connection context
0620 09:21:03.899927 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 09:21:03.900041 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":75,"threads_started":1}
I20260620 09:21:03.900835 4843 heartbeater.cc:347] Connected to a master server at 127.4.59.62:38059
I20260620 09:21:03.900941 4843 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:03.901118 4843 heartbeater.cc:511] Master 127.4.59.62:38059 requested a full tablet report, sending...
I20260620 09:21:03.901566 4384 ts_manager.cc:194] Registered new tserver with Master: e95e6893457641b2baeae824455ee646 (127.4.59.3:33379)
I20260620 09:21:03.901913 4384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:58555
I20260620 09:21:03.907702 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 09:21:03 dist-test-slave-414w krb5kdc[4339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947263, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 09:21:03 dist-test-slave-414w krb5kdc[4339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947263, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.59.1@KRBTEST.COM
I20260620 09:21:03.935379 4850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:03.925115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40294 (local address 127.4.59.1:37009)
0620 09:21:03.925289 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:03.925295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:03.925307 (+ 12us) server_negotiation.cc:407] Connection header received
0620 09:21:03.925344 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:03.925346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:03.925359 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:03.925424 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:03.926198 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.927508 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.928933 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.929482 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.929564 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:03.932191 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:03.932200 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:03.932200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:03.932208 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:03.933941 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:03.934305 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:03.934309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:03.934310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:03.934369 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:03.934577 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:03.934578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:03.934578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:03.934719 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:03.935073 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:03.935159 (+ 86us) server_negotiation.cc:299] Negotiation successful
0620 09:21:03.935265 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
W20260620 09:21:03.935719 4489 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40294
Jun 20 09:21:03 dist-test-slave-414w krb5kdc[4339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 09:21:03 dist-test-slave-414w krb5kdc[4339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.1@KRBTEST.COM
I20260620 09:21:03.958953 4850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:03.952000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40304 (local address 127.4.59.1:37009)
0620 09:21:03.952061 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:03.952063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:03.952092 (+ 29us) server_negotiation.cc:407] Connection header received
0620 09:21:03.952164 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:03.952165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:03.952178 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:03.952230 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:03.952786 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.953336 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.954078 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:03.954275 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:03.954324 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:03.956588 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:03.956590 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:03.956590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:03.956596 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:03.958007 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:03.958339 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:03.958340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:03.958340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:03.958395 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:03.958574 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:03.958574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:03.958575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:03.958669 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:03.958759 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:03.958823 (+ 64us) server_negotiation.cc:299] Negotiation successful
0620 09:21:03.958885 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 09:21:03.959501 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 4446
I20260620 09:21:03.964082 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 4580
I20260620 09:21:03.969934 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 4714
I20260620 09:21:03.974797 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 4352
2026-06-20T09:21:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1916 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:21:04 dist-test-slave-414w krb5kdc[4857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:21:04 dist-test-slave-414w krb5kdc[4857](info): set up 2 sockets
Jun 20 09:21:04 dist-test-slave-414w krb5kdc[4857](info): commencing operation
krb5kdc: starting...
W20260620 09:21:05.025305 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:21:05 dist-test-slave-414w krb5kdc[4857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:21:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:21:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:21:05.139719 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:35997
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:43115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:35997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:05.218379 4870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:05.218634 4870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:05.218669 4870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:05.219913 4870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:05.219985 4870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:05.220009 4870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:05.220031 4870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:05.220059 4870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:21:05.221532 4870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:43115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:35997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:35997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:05.221817 4870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:05.222049 4870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:05.227026 4878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:05.227026 4875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:05.227094 4876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:05.227352 4870 server_base.cc:1034] running on GCE node
I20260620 09:21:05.227602 4870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:05.227891 4870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:05.229092 4870 hybrid_clock.cc:648] HybridClock initialized: now 1781947265229086 us; error 37 us; skew 500 ppm
Jun 20 09:21:05 dist-test-slave-414w krb5kdc[4857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947265, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:05.231729 4870 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:21:05.232115 4870 webserver.cc:460] Webserver started at http://127.4.59.62:45361/ using document root <none> and password file <none>
I20260620 09:21:05.232306 4870 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:05.232347 4870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:05.232431 4870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:05.233726 4870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "8b0c619841424a45b210ec0e6420d623"
format_stamp: "Formatted at 2026-06-20 09:21:05 on dist-test-slave-414w"
server_key: "65ac18fc20f0396483dc40a7594c39f9"
server_key_iv: "4dc1407412ca8908aa54d9fdd0a32090"
server_key_version: "encryptionkey@0"
I20260620 09:21:05.234067 4870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "8b0c619841424a45b210ec0e6420d623"
format_stamp: "Formatted at 2026-06-20 09:21:05 on dist-test-slave-414w"
server_key: "65ac18fc20f0396483dc40a7594c39f9"
server_key_iv: "4dc1407412ca8908aa54d9fdd0a32090"
server_key_version: "encryptionkey@0"
I20260620 09:21:05.235320 4870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:21:05.236003 4885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:05.236207 4870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:05.236287 4870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "8b0c619841424a45b210ec0e6420d623"
format_stamp: "Formatted at 2026-06-20 09:21:05 on dist-test-slave-414w"
server_key: "65ac18fc20f0396483dc40a7594c39f9"
server_key_iv: "4dc1407412ca8908aa54d9fdd0a32090"
server_key_version: "encryptionkey@0"
I20260620 09:21:05.236341 4870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:05.250093 4870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:05.252341 4870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:05.252452 4870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:05.256179 4870 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:35997
I20260620 09:21:05.256237 4937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:35997 every 8 connection(s)
I20260620 09:21:05.256572 4870 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:21:05.257535 4938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:05.259858 4938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623: Bootstrap starting.
I20260620 09:21:05.260440 4938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:05.260735 4938 log.cc:824] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:05.261549 4938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623: No bootstrap required, opened a new log
I20260620 09:21:05.263108 4938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0c619841424a45b210ec0e6420d623" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35997 } }
I20260620 09:21:05.263263 4938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:05.263309 4938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b0c619841424a45b210ec0e6420d623, State: Initialized, Role: FOLLOWER
I20260620 09:21:05.263569 4938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0c619841424a45b210ec0e6420d623" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35997 } }
I20260620 09:21:05.263679 4938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:21:05.263724 4938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:21:05.263779 4938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:05.264031 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 4870
I20260620 09:21:05.264119 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:21:05.264263 4332 external_mini_cluster.cc:1428] Setting key 4f8632d60ada134ea9f66a8d736613d3
I20260620 09:21:05.264551 4938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0c619841424a45b210ec0e6420d623" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35997 } }
I20260620 09:21:05.264667 4938 leader_election.cc:302] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b0c619841424a45b210ec0e6420d623; no voters:
I20260620 09:21:05.264864 4938 leader_election.cc:288] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:21:05.265082 4942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:05.265132 4938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:05.265244 4942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [term 1 LEADER]: Becoming Leader. State: Replica: 8b0c619841424a45b210ec0e6420d623, State: Running, Role: LEADER
I20260620 09:21:05.265336 4942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0c619841424a45b210ec0e6420d623" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35997 } }
I20260620 09:21:05.265767 4942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b0c619841424a45b210ec0e6420d623. Latest consensus state: current_term: 1 leader_uuid: "8b0c619841424a45b210ec0e6420d623" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0c619841424a45b210ec0e6420d623" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35997 } } }
I20260620 09:21:05.265864 4942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:05.266078 4952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:21:05.266259 4944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b0c619841424a45b210ec0e6420d623" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0c619841424a45b210ec0e6420d623" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35997 } } }
I20260620 09:21:05.266324 4944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:05.266597 4952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:21:05.268123 4952 catalog_manager.cc:1347] Generated new cluster ID: 8747b381c1f844b2b20fa0f794141127
I20260620 09:21:05.268173 4952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 09:21:05 dist-test-slave-414w krb5kdc[4857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:05.275240 4952 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:21:05.275887 4952 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:21:05.276780 4943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:05.264808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37930 (local address 127.4.59.62:35997)
0620 09:21:05.265106 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:05.265116 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:05.265258 (+ 142us) server_negotiation.cc:407] Connection header received
0620 09:21:05.265379 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:05.265386 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:05.265547 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:05.265707 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:05.266523 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.267966 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.269056 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.269651 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.269755 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:05.272487 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:05.272492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:05.272492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:05.272506 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:05.275092 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:05.275499 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:05.275502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:05.275502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:05.275605 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:05.275841 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:05.275842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:05.275842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:05.276157 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:05.276289 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:05.276433 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 09:21:05.276569 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":86,"threads_started":1}
I20260620 09:21:05.290777 4952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b0c619841424a45b210ec0e6420d623: Generated new TSK 0
I20260620 09:21:05.290977 4952 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:21:05.321022 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35997
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:43115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:05.417727 4964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:05.417922 4964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:05.417946 4964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:05.419131 4964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:05.419173 4964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:05.419200 4964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:21:05.420742 4964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:43115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35997
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:05.420984 4964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:05.421252 4964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:05.426259 4969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:05.426287 4972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:05.426275 4970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:05.426546 4964 server_base.cc:1034] running on GCE node
I20260620 09:21:05.426755 4964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:05.426939 4964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:05.428102 4964 hybrid_clock.cc:648] HybridClock initialized: now 1781947265428081 us; error 34 us; skew 500 ppm
Jun 20 09:21:05 dist-test-slave-414w krb5kdc[4857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947265, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:05.430915 4964 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:21:05.431339 4964 webserver.cc:460] Webserver started at http://127.4.59.1:36159/ using document root <none> and password file <none>
I20260620 09:21:05.431528 4964 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:05.431567 4964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:05.431651 4964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:05.432611 4964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "75ad4d7417a44c82a8f023e1eaf29a1e"
format_stamp: "Formatted at 2026-06-20 09:21:05 on dist-test-slave-414w"
server_key: "c96138db7b66ee97f2e11f0e59360fd6"
server_key_iv: "96541e000782ffebe1b321c4baf0cfbe"
server_key_version: "encryptionkey@0"
I20260620 09:21:05.432897 4964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "75ad4d7417a44c82a8f023e1eaf29a1e"
format_stamp: "Formatted at 2026-06-20 09:21:05 on dist-test-slave-414w"
server_key: "c96138db7b66ee97f2e11f0e59360fd6"
server_key_iv: "96541e000782ffebe1b321c4baf0cfbe"
server_key_version: "encryptionkey@0"
I20260620 09:21:05.434173 4964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:21:05.434835 4979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:05.435009 4964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:21:05.435109 4964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "75ad4d7417a44c82a8f023e1eaf29a1e"
format_stamp: "Formatted at 2026-06-20 09:21:05 on dist-test-slave-414w"
server_key: "c96138db7b66ee97f2e11f0e59360fd6"
server_key_iv: "96541e000782ffebe1b321c4baf0cfbe"
server_key_version: "encryptionkey@0"
I20260620 09:21:05.435182 4964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:05.444864 4964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:05.446646 4964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:05.446805 4964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:05.447007 4964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:05.447285 4964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:05.447340 4964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:05.447381 4964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:05.447407 4964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:05.452996 4964 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:44231
I20260620 09:21:05.453086 5092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:44231 every 8 connection(s)
I20260620 09:21:05.453403 4964 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:21:05.457633 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 4964
I20260620 09:21:05.457723 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:21:05.457875 4332 external_mini_cluster.cc:1428] Setting key e34b12f1514cc4bdd8cb3524731c25fc
Jun 20 09:21:05 dist-test-slave-414w krb5kdc[4857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947265, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:05.463106 4943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:05.454108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:40943 (local address 127.4.59.62:35997)
0620 09:21:05.454172 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:05.454174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:05.454397 (+ 223us) server_negotiation.cc:407] Connection header received
0620 09:21:05.455278 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:05.455279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:05.455292 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:05.455350 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:05.455888 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.456379 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.457497 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.457834 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.457931 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:05.460323 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:05.460325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:05.460325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:05.460332 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:05.461735 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:05.462158 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:05.462159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:05.462160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:05.462219 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:05.462474 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:05.462474 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:05.462475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:05.462588 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:05.462663 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:05.462931 (+ 268us) server_negotiation.cc:299] Negotiation successful
0620 09:21:05.463014 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 09:21:05.463369 5095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:05.454091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35997 (local address 127.4.59.1:40943)
0620 09:21:05.454250 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:05.454288 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:05.455197 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:05.455402 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:05.455405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:05.455639 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:05.455814 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:05.455818 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.456465 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.456466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:05.457390 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:05.457392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.457959 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.457961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:05.458084 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:05.458651 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:05.458659 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:05.460224 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:05.461831 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:05.461838 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:05.461840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:05.462066 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:05.462285 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:05.462286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:05.462286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:05.462404 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:05.462655 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:05.462660 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:05.462803 (+ 143us) client_negotiation.cc:769] Sending connection context
0620 09:21:05.462911 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 09:21:05.463067 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260620 09:21:05.464166 5093 heartbeater.cc:347] Connected to a master server at 127.4.59.62:35997
I20260620 09:21:05.464284 5093 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:05.464466 5093 heartbeater.cc:511] Master 127.4.59.62:35997 requested a full tablet report, sending...
I20260620 09:21:05.465112 4902 ts_manager.cc:194] Registered new tserver with Master: 75ad4d7417a44c82a8f023e1eaf29a1e (127.4.59.1:44231)
I20260620 09:21:05.465794 4902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:40943
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:21:05.484649 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35997
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:43115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:05.569445 5098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:05.569643 5098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:05.569666 5098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:05.570739 5098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:05.570770 5098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:05.570796 5098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:21:05.572409 5098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:43115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35997
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:05.572635 5098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:05.572811 5098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:05.578281 5104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:05.578339 5106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:05.578375 5103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:05.578626 5098 server_base.cc:1034] running on GCE node
I20260620 09:21:05.578811 5098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:05.579080 5098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:05.580224 5098 hybrid_clock.cc:648] HybridClock initialized: now 1781947265580190 us; error 50 us; skew 500 ppm
Jun 20 09:21:05 dist-test-slave-414w krb5kdc[4857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947265, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:05.582836 5098 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:21:05.583299 5098 webserver.cc:460] Webserver started at http://127.4.59.2:43091/ using document root <none> and password file <none>
I20260620 09:21:05.583734 5098 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:05.583828 5098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:05.583986 5098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:05.585037 5098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "093cf41537f24f83a7c32eca58158ab8"
format_stamp: "Formatted at 2026-06-20 09:21:05 on dist-test-slave-414w"
server_key: "da2ea51caf30835bc8556ce7e8d9f43a"
server_key_iv: "16472fb1256443087f43f541bed53bdd"
server_key_version: "encryptionkey@0"
I20260620 09:21:05.585455 5098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "093cf41537f24f83a7c32eca58158ab8"
format_stamp: "Formatted at 2026-06-20 09:21:05 on dist-test-slave-414w"
server_key: "da2ea51caf30835bc8556ce7e8d9f43a"
server_key_iv: "16472fb1256443087f43f541bed53bdd"
server_key_version: "encryptionkey@0"
I20260620 09:21:05.586761 5098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:05.587615 5113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:05.587968 5098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:05.588104 5098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "093cf41537f24f83a7c32eca58158ab8"
format_stamp: "Formatted at 2026-06-20 09:21:05 on dist-test-slave-414w"
server_key: "da2ea51caf30835bc8556ce7e8d9f43a"
server_key_iv: "16472fb1256443087f43f541bed53bdd"
server_key_version: "encryptionkey@0"
I20260620 09:21:05.588179 5098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:05.599608 5098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:05.601595 5098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:05.601749 5098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:05.601966 5098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:05.602259 5098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:05.602311 5098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:05.602361 5098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:05.602389 5098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:05.608377 5098 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:42769
I20260620 09:21:05.608428 5226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:42769 every 8 connection(s)
I20260620 09:21:05.608717 5098 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:21:05.609598 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 5098
I20260620 09:21:05.609709 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:21:05.609874 4332 external_mini_cluster.cc:1428] Setting key f0048f36851aa971e27f46cdc2f3de10
Jun 20 09:21:05 dist-test-slave-414w krb5kdc[4857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947265, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
I20260620 09:21:05.618755 4943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:05.609448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:44865 (local address 127.4.59.62:35997)
0620 09:21:05.609534 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:05.609536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:05.609678 (+ 142us) server_negotiation.cc:407] Connection header received
0620 09:21:05.610810 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:05.610811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:05.610823 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:05.610886 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:05.611451 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.612137 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.613180 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.613394 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.613481 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:05.615785 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:05.615786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:05.615787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:05.615794 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:05.617329 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:05.617762 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:05.617763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:05.617763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:05.617834 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:05.618081 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:05.618081 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:05.618082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:05.618198 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:05.618284 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:05.618583 (+ 299us) server_negotiation.cc:299] Negotiation successful
0620 09:21:05.618663 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 09:21:05.618932 5229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:05.609388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35997 (local address 127.4.59.2:44865)
0620 09:21:05.609545 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:05.609585 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:05.610712 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:05.610947 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:05.610950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:05.611186 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:05.611360 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:05.611364 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.612249 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.612250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:05.613070 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:05.613072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.613492 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.613493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:05.613621 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:05.614105 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:05.614113 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:05.615689 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:05.617429 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:05.617436 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:05.617438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:05.617689 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:05.617899 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:05.617900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:05.617900 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:05.618011 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:05.618267 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:05.618274 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:05.618414 (+ 140us) client_negotiation.cc:769] Sending connection context
0620 09:21:05.618517 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 09:21:05.618668 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260620 09:21:05.619439 5227 heartbeater.cc:347] Connected to a master server at 127.4.59.62:35997
I20260620 09:21:05.619526 5227 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:05.619707 5227 heartbeater.cc:511] Master 127.4.59.62:35997 requested a full tablet report, sending...
I20260620 09:21:05.620128 4902 ts_manager.cc:194] Registered new tserver with Master: 093cf41537f24f83a7c32eca58158ab8 (127.4.59.2:42769)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
I20260620 09:21:05.620502 4902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:44865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:21:05.635406 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35997
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:43115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:05.713968 5232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:05.714159 5232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:05.714183 5232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:05.715339 5232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:05.715378 5232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:05.715405 5232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:21:05.716856 5232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:43115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35997
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:05.717090 5232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:05.717374 5232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:05.722378 5238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:05.722504 5232 server_base.cc:1034] running on GCE node
W20260620 09:21:05.722404 5240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:05.722395 5237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:05.722770 5232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:05.723070 5232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:05.724234 5232 hybrid_clock.cc:648] HybridClock initialized: now 1781947265724230 us; error 36 us; skew 500 ppm
Jun 20 09:21:05 dist-test-slave-414w krb5kdc[4857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947265, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:05.726645 5232 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:21:05.727118 5232 webserver.cc:460] Webserver started at http://127.4.59.3:44901/ using document root <none> and password file <none>
I20260620 09:21:05.727336 5232 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:05.727401 5232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:05.727553 5232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:05.728565 5232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "3e5239c560f0431daf9daa07c1541427"
format_stamp: "Formatted at 2026-06-20 09:21:05 on dist-test-slave-414w"
server_key: "19f46eb1ff053ff3240d888376eb54cc"
server_key_iv: "35844decd148c3321a552808a86d6ad8"
server_key_version: "encryptionkey@0"
I20260620 09:21:05.728915 5232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "3e5239c560f0431daf9daa07c1541427"
format_stamp: "Formatted at 2026-06-20 09:21:05 on dist-test-slave-414w"
server_key: "19f46eb1ff053ff3240d888376eb54cc"
server_key_iv: "35844decd148c3321a552808a86d6ad8"
server_key_version: "encryptionkey@0"
I20260620 09:21:05.730242 5232 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:21:05.730939 5247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:05.731112 5232 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:05.731205 5232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "3e5239c560f0431daf9daa07c1541427"
format_stamp: "Formatted at 2026-06-20 09:21:05 on dist-test-slave-414w"
server_key: "19f46eb1ff053ff3240d888376eb54cc"
server_key_iv: "35844decd148c3321a552808a86d6ad8"
server_key_version: "encryptionkey@0"
I20260620 09:21:05.731290 5232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:05.755529 5232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:05.757700 5232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:05.757874 5232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:05.758121 5232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:05.758466 5232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:05.758519 5232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:05.758553 5232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:05.758584 5232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:05.764683 5232 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:36395
I20260620 09:21:05.764703 5360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:36395 every 8 connection(s)
I20260620 09:21:05.765070 5232 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:21:05.769910 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 5232
I20260620 09:21:05.770002 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:21:05.770164 4332 external_mini_cluster.cc:1428] Setting key 33de449bd52f15d90e27a2a95cc17ee6
Jun 20 09:21:05 dist-test-slave-414w krb5kdc[4857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947265, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:05.775199 4943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:05.765914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:48929 (local address 127.4.59.62:35997)
0620 09:21:05.765983 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:05.765985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:05.766190 (+ 205us) server_negotiation.cc:407] Connection header received
0620 09:21:05.767230 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:05.767231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:05.767243 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:05.767294 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:05.767932 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.768431 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.769549 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.769789 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.769863 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:05.772556 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:05.772557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:05.772557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:05.772563 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:05.773946 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:05.774310 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:05.774311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:05.774311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:05.774360 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:05.774672 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:05.774673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:05.774673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:05.774780 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:05.774836 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:05.775076 (+ 240us) server_negotiation.cc:299] Negotiation successful
0620 09:21:05.775135 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 09:21:05.775418 5363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:05.765909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35997 (local address 127.4.59.3:48929)
0620 09:21:05.766055 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:05.766098 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:05.767122 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:05.767377 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:05.767380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:05.767641 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:05.767831 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:05.767834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.768524 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.768525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:05.769434 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:05.769436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.769877 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.769878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:05.770023 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:05.770773 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:05.770782 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:05.772446 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:05.774020 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:05.774026 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:05.774027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:05.774256 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:05.774423 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:05.774424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:05.774424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:05.774596 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:05.774841 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:05.774846 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:05.774967 (+ 121us) client_negotiation.cc:769] Sending connection context
0620 09:21:05.775048 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 09:21:05.775164 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260620 09:21:05.775934 5361 heartbeater.cc:347] Connected to a master server at 127.4.59.62:35997
I20260620 09:21:05.776027 5361 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:05.776157 5361 heartbeater.cc:511] Master 127.4.59.62:35997 requested a full tablet report, sending...
I20260620 09:21:05.776508 4902 ts_manager.cc:194] Registered new tserver with Master: 3e5239c560f0431daf9daa07c1541427 (127.4.59.3:36395)
I20260620 09:21:05.776885 4902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:48929
I20260620 09:21:05.781785 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:21:05.789124 4943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:05.783242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37936 (local address 127.4.59.62:35997)
0620 09:21:05.783304 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:05.783307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:05.783392 (+ 85us) server_negotiation.cc:407] Connection header received
0620 09:21:05.783464 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:05.783465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:05.783475 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:05.783568 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:05.784096 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.784561 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.785322 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.785510 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.785591 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:05.786727 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:05.786728 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:05.786729 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:05.786735 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:05.787937 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:05.788316 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:05.788316 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:05.788317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:05.788378 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:05.788589 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:05.788590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:05.788590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:05.788730 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:05.788850 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:05.788951 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 09:21:05.789025 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 09:21:05.791465 4902 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37936:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:21:05.793810 4902 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:21:05.802367 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:05.797681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:42769 (local address 127.0.0.1:58638)
0620 09:21:05.798251 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:05.798261 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:05.798318 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:05.798687 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:05.798690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:05.798692 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:05.798759 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:05.798761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.800015 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.800016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:05.801307 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:05.801310 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.802073 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.802074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:05.802156 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:05.802169 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 09:21:05.802240 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 09:21:05.802294 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":93,"threads_started":1}
I20260620 09:21:05.802413 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:05.798091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58638 (local address 127.4.59.2:42769)
0620 09:21:05.798446 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:05.798450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:05.798459 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:21:05.798495 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:05.798498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:05.798513 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:05.798587 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:05.798841 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.799908 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.801391 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.801954 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.802069 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:05.802130 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:05.802258 (+ 128us) server_negotiation.cc:299] Negotiation successful
0620 09:21:05.802296 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":76,"threads_started":1}
I20260620 09:21:05.803036 5161 tablet_service.cc:1467] Processing CreateTablet for tablet c9a22e2372834317bee408a3d1ac6ad2 (DEFAULT_TABLE table=test-table [id=2d77c550cf2f444e8e82a696eeccb6a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:05.803356 5161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9a22e2372834317bee408a3d1ac6ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:05.803352 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:05.797818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:36395 (local address 127.0.0.1:46762)
0620 09:21:05.798283 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:05.798294 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:05.798351 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:05.799097 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:05.799097 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:05.799100 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:05.799163 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:05.799165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.800745 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.800745 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:05.801900 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:05.801903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.803134 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.803135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:05.803192 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:05.803196 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:21:05.803242 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 09:21:05.803282 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":101,"threads_started":1}
I20260620 09:21:05.803591 5375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:05.797918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46762 (local address 127.4.59.3:36395)
0620 09:21:05.798123 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:05.798128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:05.798406 (+ 278us) server_negotiation.cc:407] Connection header received
0620 09:21:05.798460 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:05.798462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:05.798478 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:05.798988 (+ 510us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:05.799228 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.800349 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.802001 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.803003 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.803056 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:05.803105 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:05.803472 (+ 367us) server_negotiation.cc:299] Negotiation successful
0620 09:21:05.803513 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":80,"threads_started":1}
I20260620 09:21:05.803767 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:05.797666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:44231 (local address 127.0.0.1:33874)
0620 09:21:05.798055 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:05.798096 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:05.798584 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:05.799430 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:05.799431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:05.799434 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:05.799508 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:05.799509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.801236 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.801237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:05.802295 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:05.802297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.803559 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.803560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:05.803624 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:05.803628 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:21:05.803677 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 09:21:05.803713 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":57,"threads_started":1}
I20260620 09:21:05.803996 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:05.797904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33874 (local address 127.4.59.1:44231)
0620 09:21:05.798203 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:05.798207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:05.798218 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:21:05.798702 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:05.798705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:05.798763 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:05.798891 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:05.799687 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.801122 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.802380 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:05.803401 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:05.803477 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:05.803528 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:05.803875 (+ 347us) server_negotiation.cc:299] Negotiation successful
0620 09:21:05.803923 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":110,"threads_started":1}
I20260620 09:21:05.804613 5027 tablet_service.cc:1467] Processing CreateTablet for tablet c9a22e2372834317bee408a3d1ac6ad2 (DEFAULT_TABLE table=test-table [id=2d77c550cf2f444e8e82a696eeccb6a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:05.804960 5027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9a22e2372834317bee408a3d1ac6ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:05.806886 5378 tablet_bootstrap.cc:492] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8: Bootstrap starting.
I20260620 09:21:05.807511 5378 tablet_bootstrap.cc:654] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:05.807538 5295 tablet_service.cc:1467] Processing CreateTablet for tablet c9a22e2372834317bee408a3d1ac6ad2 (DEFAULT_TABLE table=test-table [id=2d77c550cf2f444e8e82a696eeccb6a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:05.807857 5295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9a22e2372834317bee408a3d1ac6ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:05.808125 5378 log.cc:824] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:05.808495 5380 tablet_bootstrap.cc:492] T c9a22e2372834317bee408a3d1ac6ad2 P 75ad4d7417a44c82a8f023e1eaf29a1e: Bootstrap starting.
I20260620 09:21:05.808924 5378 tablet_bootstrap.cc:492] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8: No bootstrap required, opened a new log
I20260620 09:21:05.808998 5378 ts_tablet_manager.cc:1397] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:21:05.809302 5380 tablet_bootstrap.cc:654] T c9a22e2372834317bee408a3d1ac6ad2 P 75ad4d7417a44c82a8f023e1eaf29a1e: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:05.809839 5380 log.cc:824] T c9a22e2372834317bee408a3d1ac6ad2 P 75ad4d7417a44c82a8f023e1eaf29a1e: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:05.810583 5380 tablet_bootstrap.cc:492] T c9a22e2372834317bee408a3d1ac6ad2 P 75ad4d7417a44c82a8f023e1eaf29a1e: No bootstrap required, opened a new log
I20260620 09:21:05.810654 5380 ts_tablet_manager.cc:1397] T c9a22e2372834317bee408a3d1ac6ad2 P 75ad4d7417a44c82a8f023e1eaf29a1e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:21:05.810935 5382 tablet_bootstrap.cc:492] T c9a22e2372834317bee408a3d1ac6ad2 P 3e5239c560f0431daf9daa07c1541427: Bootstrap starting.
I20260620 09:21:05.811129 5378 raft_consensus.cc:348] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ad4d7417a44c82a8f023e1eaf29a1e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 44231 } } peers { permanent_uuid: "3e5239c560f0431daf9daa07c1541427" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36395 } } peers { permanent_uuid: "093cf41537f24f83a7c32eca58158ab8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42769 } }
I20260620 09:21:05.811321 5378 raft_consensus.cc:374] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:05.811359 5378 raft_consensus.cc:729] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093cf41537f24f83a7c32eca58158ab8, State: Initialized, Role: FOLLOWER
I20260620 09:21:05.811497 5378 consensus_queue.cc:260] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ad4d7417a44c82a8f023e1eaf29a1e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 44231 } } peers { permanent_uuid: "3e5239c560f0431daf9daa07c1541427" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36395 } } peers { permanent_uuid: "093cf41537f24f83a7c32eca58158ab8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42769 } }
I20260620 09:21:05.811561 5382 tablet_bootstrap.cc:654] T c9a22e2372834317bee408a3d1ac6ad2 P 3e5239c560f0431daf9daa07c1541427: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:05.811760 5378 ts_tablet_manager.cc:1428] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:21:05.811880 5227 heartbeater.cc:503] Master 127.4.59.62:35997 was elected leader, sending a full tablet report...
I20260620 09:21:05.812111 5382 log.cc:824] T c9a22e2372834317bee408a3d1ac6ad2 P 3e5239c560f0431daf9daa07c1541427: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:05.812640 5380 raft_consensus.cc:348] T c9a22e2372834317bee408a3d1ac6ad2 P 75ad4d7417a44c82a8f023e1eaf29a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ad4d7417a44c82a8f023e1eaf29a1e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 44231 } } peers { permanent_uuid: "3e5239c560f0431daf9daa07c1541427" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36395 } } peers { permanent_uuid: "093cf41537f24f83a7c32eca58158ab8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42769 } }
I20260620 09:21:05.812776 5382 tablet_bootstrap.cc:492] T c9a22e2372834317bee408a3d1ac6ad2 P 3e5239c560f0431daf9daa07c1541427: No bootstrap required, opened a new log
I20260620 09:21:05.812815 5380 raft_consensus.cc:374] T c9a22e2372834317bee408a3d1ac6ad2 P 75ad4d7417a44c82a8f023e1eaf29a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:05.812841 5380 raft_consensus.cc:729] T c9a22e2372834317bee408a3d1ac6ad2 P 75ad4d7417a44c82a8f023e1eaf29a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ad4d7417a44c82a8f023e1eaf29a1e, State: Initialized, Role: FOLLOWER
I20260620 09:21:05.812857 5382 ts_tablet_manager.cc:1397] T c9a22e2372834317bee408a3d1ac6ad2 P 3e5239c560f0431daf9daa07c1541427: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:21:05.812969 5380 consensus_queue.cc:260] T c9a22e2372834317bee408a3d1ac6ad2 P 75ad4d7417a44c82a8f023e1eaf29a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ad4d7417a44c82a8f023e1eaf29a1e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 44231 } } peers { permanent_uuid: "3e5239c560f0431daf9daa07c1541427" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36395 } } peers { permanent_uuid: "093cf41537f24f83a7c32eca58158ab8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42769 } }
I20260620 09:21:05.813267 5093 heartbeater.cc:503] Master 127.4.59.62:35997 was elected leader, sending a full tablet report...
I20260620 09:21:05.813267 5380 ts_tablet_manager.cc:1428] T c9a22e2372834317bee408a3d1ac6ad2 P 75ad4d7417a44c82a8f023e1eaf29a1e: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260620 09:21:05.814771 5382 raft_consensus.cc:348] T c9a22e2372834317bee408a3d1ac6ad2 P 3e5239c560f0431daf9daa07c1541427 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ad4d7417a44c82a8f023e1eaf29a1e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 44231 } } peers { permanent_uuid: "3e5239c560f0431daf9daa07c1541427" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36395 } } peers { permanent_uuid: "093cf41537f24f83a7c32eca58158ab8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42769 } }
I20260620 09:21:05.814922 5382 raft_consensus.cc:374] T c9a22e2372834317bee408a3d1ac6ad2 P 3e5239c560f0431daf9daa07c1541427 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:05.814967 5382 raft_consensus.cc:729] T c9a22e2372834317bee408a3d1ac6ad2 P 3e5239c560f0431daf9daa07c1541427 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5239c560f0431daf9daa07c1541427, State: Initialized, Role: FOLLOWER
I20260620 09:21:05.815083 5382 consensus_queue.cc:260] T c9a22e2372834317bee408a3d1ac6ad2 P 3e5239c560f0431daf9daa07c1541427 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ad4d7417a44c82a8f023e1eaf29a1e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 44231 } } peers { permanent_uuid: "3e5239c560f0431daf9daa07c1541427" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36395 } } peers { permanent_uuid: "093cf41537f24f83a7c32eca58158ab8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42769 } }
I20260620 09:21:05.815354 5382 ts_tablet_manager.cc:1428] T c9a22e2372834317bee408a3d1ac6ad2 P 3e5239c560f0431daf9daa07c1541427: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 09:21:05.815392 5361 heartbeater.cc:503] Master 127.4.59.62:35997 was elected leader, sending a full tablet report...
W20260620 09:21:05.859246 5228 tablet.cc:2369] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 09:21:05.954174 5094 tablet.cc:2369] T c9a22e2372834317bee408a3d1ac6ad2 P 75ad4d7417a44c82a8f023e1eaf29a1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:21:06.007359 5383 raft_consensus.cc:482] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:21:06.007485 5383 raft_consensus.cc:504] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ad4d7417a44c82a8f023e1eaf29a1e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 44231 } } peers { permanent_uuid: "3e5239c560f0431daf9daa07c1541427" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36395 } } peers { permanent_uuid: "093cf41537f24f83a7c32eca58158ab8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42769 } }
I20260620 09:21:06.007850 5383 leader_election.cc:288] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ad4d7417a44c82a8f023e1eaf29a1e (127.4.59.1:44231), 3e5239c560f0431daf9daa07c1541427 (127.4.59.3:36395)
I20260620 09:21:06.011967 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:06.008169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:46683 (local address 127.4.59.1:44231)
0620 09:21:06.008214 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:06.008216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:06.008396 (+ 180us) server_negotiation.cc:407] Connection header received
0620 09:21:06.008426 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:06.008427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:06.008445 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:06.008487 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:06.008702 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:06.009458 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:06.011026 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:06.011585 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:06.011664 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:06.011674 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:06.011862 (+ 188us) server_negotiation.cc:299] Negotiation successful
0620 09:21:06.011898 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 09:21:06.011969 5375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:06.008127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:38895 (local address 127.4.59.3:36395)
0620 09:21:06.008188 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:06.008190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:06.008203 (+ 13us) server_negotiation.cc:407] Connection header received
0620 09:21:06.008237 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:06.008238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:06.008250 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:06.008343 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:06.008681 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:06.009464 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:06.011026 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:06.011595 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:06.011664 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:06.011674 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:06.011862 (+ 188us) server_negotiation.cc:299] Negotiation successful
0620 09:21:06.011901 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 09:21:06.011966 5229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:06.008044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:36395 (local address 127.4.59.2:38895)
0620 09:21:06.008116 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:06.008128 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:06.008179 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:06.008404 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:06.008405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:06.008407 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:06.008574 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:06.008576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:06.009566 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:06.009567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:06.010917 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:06.010921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:06.011680 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:06.011681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:06.011766 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:06.011768 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:21:06.011829 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 09:21:06.011878 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":56}
I20260620 09:21:06.011976 5387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:06.008046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:44231 (local address 127.4.59.2:46683)
0620 09:21:06.008293 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:06.008320 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:06.008373 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:06.008563 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:06.008565 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:06.008567 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:06.008629 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:06.008630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:06.009579 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:06.009579 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:06.010917 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:06.010921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:06.011680 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:06.011681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:06.011762 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:06.011767 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 09:21:06.011829 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 09:21:06.011878 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":148,"thread_start_us":105,"threads_started":1}
I20260620 09:21:06.012359 5047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9a22e2372834317bee408a3d1ac6ad2" candidate_uuid: "093cf41537f24f83a7c32eca58158ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ad4d7417a44c82a8f023e1eaf29a1e" is_pre_election: true
I20260620 09:21:06.012359 5315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9a22e2372834317bee408a3d1ac6ad2" candidate_uuid: "093cf41537f24f83a7c32eca58158ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5239c560f0431daf9daa07c1541427" is_pre_election: true
I20260620 09:21:06.012514 5047 raft_consensus.cc:2454] T c9a22e2372834317bee408a3d1ac6ad2 P 75ad4d7417a44c82a8f023e1eaf29a1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 093cf41537f24f83a7c32eca58158ab8 in term 0.
I20260620 09:21:06.012528 5315 raft_consensus.cc:2454] T c9a22e2372834317bee408a3d1ac6ad2 P 3e5239c560f0431daf9daa07c1541427 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 093cf41537f24f83a7c32eca58158ab8 in term 0.
I20260620 09:21:06.012761 5114 leader_election.cc:302] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093cf41537f24f83a7c32eca58158ab8, 75ad4d7417a44c82a8f023e1eaf29a1e; no voters:
I20260620 09:21:06.012959 5383 raft_consensus.cc:2781] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:21:06.013032 5383 raft_consensus.cc:482] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:21:06.013067 5383 raft_consensus.cc:3037] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:06.013744 5383 raft_consensus.cc:504] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ad4d7417a44c82a8f023e1eaf29a1e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 44231 } } peers { permanent_uuid: "3e5239c560f0431daf9daa07c1541427" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36395 } } peers { permanent_uuid: "093cf41537f24f83a7c32eca58158ab8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42769 } }
I20260620 09:21:06.013900 5383 leader_election.cc:288] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [CANDIDATE]: Term 1 election: Requested vote from peers 75ad4d7417a44c82a8f023e1eaf29a1e (127.4.59.1:44231), 3e5239c560f0431daf9daa07c1541427 (127.4.59.3:36395)
I20260620 09:21:06.014036 5315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9a22e2372834317bee408a3d1ac6ad2" candidate_uuid: "093cf41537f24f83a7c32eca58158ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5239c560f0431daf9daa07c1541427"
I20260620 09:21:06.014109 5315 raft_consensus.cc:3037] T c9a22e2372834317bee408a3d1ac6ad2 P 3e5239c560f0431daf9daa07c1541427 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:06.014065 5047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9a22e2372834317bee408a3d1ac6ad2" candidate_uuid: "093cf41537f24f83a7c32eca58158ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ad4d7417a44c82a8f023e1eaf29a1e"
I20260620 09:21:06.014158 5047 raft_consensus.cc:3037] T c9a22e2372834317bee408a3d1ac6ad2 P 75ad4d7417a44c82a8f023e1eaf29a1e [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:06.014771 5315 raft_consensus.cc:2454] T c9a22e2372834317bee408a3d1ac6ad2 P 3e5239c560f0431daf9daa07c1541427 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 093cf41537f24f83a7c32eca58158ab8 in term 1.
I20260620 09:21:06.014920 5047 raft_consensus.cc:2454] T c9a22e2372834317bee408a3d1ac6ad2 P 75ad4d7417a44c82a8f023e1eaf29a1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 093cf41537f24f83a7c32eca58158ab8 in term 1.
I20260620 09:21:06.014933 5115 leader_election.cc:302] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093cf41537f24f83a7c32eca58158ab8, 3e5239c560f0431daf9daa07c1541427; no voters:
I20260620 09:21:06.015131 5383 raft_consensus.cc:2781] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:06.015288 5383 raft_consensus.cc:686] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [term 1 LEADER]: Becoming Leader. State: Replica: 093cf41537f24f83a7c32eca58158ab8, State: Running, Role: LEADER
I20260620 09:21:06.015384 5383 consensus_queue.cc:237] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ad4d7417a44c82a8f023e1eaf29a1e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 44231 } } peers { permanent_uuid: "3e5239c560f0431daf9daa07c1541427" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36395 } } peers { permanent_uuid: "093cf41537f24f83a7c32eca58158ab8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42769 } }
W20260620 09:21:06.015627 5362 tablet.cc:2369] T c9a22e2372834317bee408a3d1ac6ad2 P 3e5239c560f0431daf9daa07c1541427: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:21:06.016251 4901 catalog_manager.cc:5495] T c9a22e2372834317bee408a3d1ac6ad2 P 093cf41537f24f83a7c32eca58158ab8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 093cf41537f24f83a7c32eca58158ab8 (127.4.59.2). New cstate: current_term: 1 leader_uuid: "093cf41537f24f83a7c32eca58158ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ad4d7417a44c82a8f023e1eaf29a1e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 44231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5239c560f0431daf9daa07c1541427" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093cf41537f24f83a7c32eca58158ab8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42769 } health_report { overall_health: HEALTHY } } }
Jun 20 09:21:06 dist-test-slave-414w krb5kdc[4857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.1@KRBTEST.COM
I20260620 09:21:06.031818 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:06.024833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33876 (local address 127.4.59.1:44231)
0620 09:21:06.024920 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:06.024922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:06.024961 (+ 39us) server_negotiation.cc:407] Connection header received
0620 09:21:06.025005 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:06.025006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:06.025014 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:06.025064 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:06.025663 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:06.026126 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:06.026796 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:06.026986 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:06.027041 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:06.029288 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:06.029296 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:06.029297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:06.029303 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:06.030852 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:06.031187 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:06.031190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:06.031190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:06.031254 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:06.031422 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:06.031423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:06.031423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:06.031545 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:06.031644 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:06.031700 (+ 56us) server_negotiation.cc:299] Negotiation successful
0620 09:21:06.031756 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Jun 20 09:21:06 dist-test-slave-414w krb5kdc[4857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947266, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 09:21:06 dist-test-slave-414w krb5kdc[4857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947266, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.59.1@KRBTEST.COM
I20260620 09:21:06.053828 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:06.047099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33880 (local address 127.4.59.1:44231)
0620 09:21:06.047175 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:06.047176 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:06.047225 (+ 49us) server_negotiation.cc:407] Connection header received
0620 09:21:06.047283 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:06.047284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:06.047293 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:06.047334 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:06.047856 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:06.048309 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:06.048976 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:06.049234 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:06.049289 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:06.051671 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:06.051672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:06.051673 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:06.051680 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:06.052843 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:06.053197 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:06.053198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:06.053198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:06.053259 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:06.053423 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:06.053423 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:06.053423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:06.053506 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:06.053572 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:06.053704 (+ 132us) server_negotiation.cc:299] Negotiation successful
0620 09:21:06.053759 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Jun 20 09:21:06 dist-test-slave-414w krb5kdc[4857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 09:21:06 dist-test-slave-414w krb5kdc[4857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.1@KRBTEST.COM
I20260620 09:21:06.075870 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:06.068872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33894 (local address 127.4.59.1:44231)
0620 09:21:06.068940 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:06.068943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:06.069042 (+ 99us) server_negotiation.cc:407] Connection header received
0620 09:21:06.069121 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:06.069122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:06.069173 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:06.069245 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:06.069911 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:06.070391 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:06.071119 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:06.071307 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:06.071390 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:06.073524 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:06.073525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:06.073526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:06.073533 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:06.074878 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:06.075179 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:06.075180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:06.075180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:06.075227 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:06.075387 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:06.075387 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:06.075387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:06.075473 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:06.075548 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:06.075747 (+ 199us) server_negotiation.cc:299] Negotiation successful
0620 09:21:06.075803 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260620 09:21:06.076347 5007 tablet_service.cc:2908] Rejecting scan request for tablet c9a22e2372834317bee408a3d1ac6ad2: Uninitialized: clean time has not yet been initialized
I20260620 09:21:06.077282 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 4964
I20260620 09:21:06.082221 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 5098
I20260620 09:21:06.087126 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 5232
I20260620 09:21:06.091707 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 4870
2026-06-20T09:21:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2118 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:21:06 dist-test-slave-414w krb5kdc[5402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:21:06 dist-test-slave-414w krb5kdc[5402](info): set up 2 sockets
Jun 20 09:21:06 dist-test-slave-414w krb5kdc[5402](info): commencing operation
krb5kdc: starting...
W20260620 09:21:07.141180 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:21:07 dist-test-slave-414w krb5kdc[5402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:21:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:21:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:21:07.257490 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:33003
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:33003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:07.335672 5415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:07.335904 5415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:07.335942 5415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:07.337070 5415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:07.337136 5415 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 09:21:07.337204 5415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:07.337239 5415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:07.337263 5415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:07.337294 5415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:21:07.338742 5415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:33003
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:33003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:07.339010 5415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:07.339228 5415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:07.343757 5423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:07.343853 5421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:07.343873 5415 server_base.cc:1034] running on GCE node
W20260620 09:21:07.343755 5420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:07.344172 5415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:07.344434 5415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:07.345574 5415 hybrid_clock.cc:648] HybridClock initialized: now 1781947267345557 us; error 32 us; skew 500 ppm
Jun 20 09:21:07 dist-test-slave-414w krb5kdc[5402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947267, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:07.350907 5415 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:21:07.351337 5415 webserver.cc:460] Webserver started at http://127.4.59.62:42995/ using document root <none> and password file <none>
I20260620 09:21:07.351509 5415 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:07.351549 5415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:07.351632 5415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:07.352594 5415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "81210b6086ea48219e467afceb2756da"
format_stamp: "Formatted at 2026-06-20 09:21:07 on dist-test-slave-414w"
server_key: "3298549516347a47df408a545a0cde1c"
server_key_iv: "0e333f38eb8bfecc90aededb02cd1bd4"
server_key_version: "encryptionkey@0"
I20260620 09:21:07.352887 5415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "81210b6086ea48219e467afceb2756da"
format_stamp: "Formatted at 2026-06-20 09:21:07 on dist-test-slave-414w"
server_key: "3298549516347a47df408a545a0cde1c"
server_key_iv: "0e333f38eb8bfecc90aededb02cd1bd4"
server_key_version: "encryptionkey@0"
I20260620 09:21:07.354203 5415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:21:07.354920 5430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:07.355149 5415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:21:07.355227 5415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "81210b6086ea48219e467afceb2756da"
format_stamp: "Formatted at 2026-06-20 09:21:07 on dist-test-slave-414w"
server_key: "3298549516347a47df408a545a0cde1c"
server_key_iv: "0e333f38eb8bfecc90aededb02cd1bd4"
server_key_version: "encryptionkey@0"
I20260620 09:21:07.355336 5415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:07.383575 5415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:07.385565 5415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:07.385708 5415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:07.390287 5415 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:33003
I20260620 09:21:07.390337 5492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:33003 every 8 connection(s)
I20260620 09:21:07.390645 5415 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:21:07.391274 5493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:07.391707 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 5415
I20260620 09:21:07.391809 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:21:07.391973 4332 external_mini_cluster.cc:1428] Setting key 18b27ebf3c1e506df56aa07e7026f436
I20260620 09:21:07.394114 5493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da: Bootstrap starting.
I20260620 09:21:07.394716 5493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:07.394995 5493 log.cc:824] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:07.395738 5493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da: No bootstrap required, opened a new log
I20260620 09:21:07.398270 5493 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81210b6086ea48219e467afceb2756da" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33003 } }
I20260620 09:21:07.398398 5493 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:07.398420 5493 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81210b6086ea48219e467afceb2756da, State: Initialized, Role: FOLLOWER
Jun 20 09:21:07 dist-test-slave-414w krb5kdc[5402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:07.398557 5493 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81210b6086ea48219e467afceb2756da" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33003 } }
I20260620 09:21:07.398641 5493 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:21:07.398687 5493 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:21:07.398736 5493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:07.399341 5493 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81210b6086ea48219e467afceb2756da" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33003 } }
I20260620 09:21:07.399461 5493 leader_election.cc:302] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81210b6086ea48219e467afceb2756da; no voters:
I20260620 09:21:07.399631 5493 leader_election.cc:288] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:21:07.399658 5498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:07.399768 5498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [term 1 LEADER]: Becoming Leader. State: Replica: 81210b6086ea48219e467afceb2756da, State: Running, Role: LEADER
I20260620 09:21:07.399881 5493 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:07.399899 5498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81210b6086ea48219e467afceb2756da" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33003 } }
I20260620 09:21:07.400183 5498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81210b6086ea48219e467afceb2756da. Latest consensus state: current_term: 1 leader_uuid: "81210b6086ea48219e467afceb2756da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81210b6086ea48219e467afceb2756da" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33003 } } }
I20260620 09:21:07.400199 5499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81210b6086ea48219e467afceb2756da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81210b6086ea48219e467afceb2756da" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33003 } } }
I20260620 09:21:07.400247 5498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:07.400283 5499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:07.400485 5504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:21:07.400947 5504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:21:07.402733 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.392478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59258 (local address 127.4.59.62:33003)
0620 09:21:07.392640 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:07.392652 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:07.392679 (+ 27us) server_negotiation.cc:407] Connection header received
0620 09:21:07.393062 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:07.393068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:07.393110 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:07.393281 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:07.394028 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.394962 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.395929 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.396327 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.396413 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.399044 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:07.399047 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:07.399047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:07.399057 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:07.401077 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:07.401513 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:07.401516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:07.401516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:07.401611 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:07.401827 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:07.401829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:07.401829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:07.402118 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:07.402252 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:07.402397 (+ 145us) server_negotiation.cc:299] Negotiation successful
0620 09:21:07.402532 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260620 09:21:07.402833 5504 catalog_manager.cc:1347] Generated new cluster ID: e93fcda01a0e4e568bf30ae880c061a1
I20260620 09:21:07.402875 5504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:21:07.410619 5504 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:21:07.411101 5504 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:21:07.415750 5504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da: Generated new TSK 0
I20260620 09:21:07.415907 5504 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:21:07.443868 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:33003
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:07.541318 5519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:07.541512 5519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:07.541538 5519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:07.542752 5519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:07.542795 5519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:07.542816 5519 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 09:21:07.542835 5519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:21:07.544348 5519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.59.62:33003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:07.544610 5519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:07.544840 5519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:07.551119 5524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:07.551158 5527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:07.551158 5525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:07.551370 5519 server_base.cc:1034] running on GCE node
I20260620 09:21:07.551872 5519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:07.552136 5519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:07.553304 5519 hybrid_clock.cc:648] HybridClock initialized: now 1781947267553282 us; error 38 us; skew 500 ppm
Jun 20 09:21:07 dist-test-slave-414w krb5kdc[5402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947267, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:07.555917 5519 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:21:07.556417 5519 webserver.cc:460] Webserver started at http://127.4.59.1:33355/ using document root <none> and password file <none>
I20260620 09:21:07.556672 5519 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:07.556854 5519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:07.557019 5519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:07.558378 5519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "176c1ee9098d4fef911112f3b54995b3"
format_stamp: "Formatted at 2026-06-20 09:21:07 on dist-test-slave-414w"
server_key: "d12b9c145fcde4174032e2b2b5462573"
server_key_iv: "9354404787f1889f3b9ea798ad1018d8"
server_key_version: "encryptionkey@0"
I20260620 09:21:07.558809 5519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "176c1ee9098d4fef911112f3b54995b3"
format_stamp: "Formatted at 2026-06-20 09:21:07 on dist-test-slave-414w"
server_key: "d12b9c145fcde4174032e2b2b5462573"
server_key_iv: "9354404787f1889f3b9ea798ad1018d8"
server_key_version: "encryptionkey@0"
I20260620 09:21:07.560226 5519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 09:21:07.560981 5534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:07.561231 5519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 09:21:07.561332 5519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "176c1ee9098d4fef911112f3b54995b3"
format_stamp: "Formatted at 2026-06-20 09:21:07 on dist-test-slave-414w"
server_key: "d12b9c145fcde4174032e2b2b5462573"
server_key_iv: "9354404787f1889f3b9ea798ad1018d8"
server_key_version: "encryptionkey@0"
I20260620 09:21:07.561419 5519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:07.575804 5519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:07.578260 5519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:07.578424 5519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:07.579003 5519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:07.579069 5519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:07.579113 5519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:07.579140 5519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:07.586161 5519 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:39233
I20260620 09:21:07.586247 5649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:39233 every 8 connection(s)
Jun 20 09:21:07 dist-test-slave-414w krb5kdc[5402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947267, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:07.586548 5519 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:21:07.588954 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 5519
I20260620 09:21:07.589054 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:21:07.589277 4332 external_mini_cluster.cc:1428] Setting key fb01b63e75e7ce3d6a18c8989f6c0f59
I20260620 09:21:07.589444 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.579460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:48365 (local address 127.4.59.62:33003)
0620 09:21:07.579509 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:07.579511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:07.579797 (+ 286us) server_negotiation.cc:407] Connection header received
0620 09:21:07.580358 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:07.580360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:07.580373 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:07.580431 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:07.581408 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.582258 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.583933 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.584237 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.584309 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.586866 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:07.586867 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:07.586868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:07.586876 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:07.588109 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:07.588464 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:07.588464 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:07.588465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:07.588513 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:07.588741 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:07.588741 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:07.588741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:07.588839 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:07.588888 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:07.589247 (+ 359us) server_negotiation.cc:299] Negotiation successful
0620 09:21:07.589339 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:21:07.589659 5543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.579470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:33003 (local address 127.4.59.1:48365)
0620 09:21:07.579608 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:07.579642 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:07.580222 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:07.580551 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:07.580557 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:07.580951 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:07.581298 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.581305 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.582378 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.582379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.583833 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.583837 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.584345 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.584346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.584518 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.585106 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:07.585116 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:07.586760 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:07.588178 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:07.588184 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:07.588186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:07.588396 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:07.588587 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:07.588588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:07.588588 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:07.588687 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:07.588903 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:07.588911 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:07.589079 (+ 168us) client_negotiation.cc:769] Sending connection context
0620 09:21:07.589222 (+ 143us) client_negotiation.cc:240] Negotiation successful
0620 09:21:07.589391 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":82,"threads_started":1}
I20260620 09:21:07.590142 5650 heartbeater.cc:347] Connected to a master server at 127.4.59.62:33003
I20260620 09:21:07.590289 5650 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:07.590513 5650 heartbeater.cc:511] Master 127.4.59.62:33003 requested a full tablet report, sending...
I20260620 09:21:07.591107 5446 ts_manager.cc:194] Registered new tserver with Master: 176c1ee9098d4fef911112f3b54995b3 (127.4.59.1:39233)
I20260620 09:21:07.591768 5446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:48365
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
I20260620 09:21:07.598572 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.591312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:58401 (local address 127.4.59.62:33003)
0620 09:21:07.591350 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:07.591352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:07.592093 (+ 741us) server_negotiation.cc:407] Connection header received
0620 09:21:07.592139 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:07.592140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:07.592150 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:07.592206 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:07.592712 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.593515 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.594629 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.594891 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.594961 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.595981 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:07.595983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:07.595983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:07.595989 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:07.597438 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:07.597815 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:07.597816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:07.597816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:07.597865 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:07.598119 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:07.598120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:07.598120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:07.598231 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:07.598285 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:07.598453 (+ 168us) server_negotiation.cc:299] Negotiation successful
0620 09:21:07.598512 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:21:07.598574 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.591145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:33003 (local address 127.4.59.1:58401)
0620 09:21:07.592003 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:07.592014 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:07.592086 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:07.592211 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:07.592211 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:07.592420 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:07.592594 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.592599 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.593637 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.593638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.594546 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.594549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.594982 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.594983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.595081 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.595483 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:07.595489 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:07.595893 (+ 404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:07.597521 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:07.597532 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:07.597533 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:07.597738 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:07.597935 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:07.597937 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:07.597937 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:07.598009 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:07.598303 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:07.598305 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:07.598350 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 09:21:07.598422 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 09:21:07.598496 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"thread_start_us":63,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:21:07.616509 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:33003
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:07.699786 5659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:07.700016 5659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:07.700052 5659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:07.701331 5659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:07.701385 5659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:07.701422 5659 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 09:21:07.701458 5659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:21:07.702838 5659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.59.62:33003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:07.703094 5659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:07.703312 5659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 09:21:07.708174 5659 server_base.cc:1034] running on GCE node
W20260620 09:21:07.708144 5667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:07.708178 5665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:07.708145 5664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:07.708590 5659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:07.708835 5659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:07.709980 5659 hybrid_clock.cc:648] HybridClock initialized: now 1781947267709966 us; error 29 us; skew 500 ppm
Jun 20 09:21:07 dist-test-slave-414w krb5kdc[5402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947267, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:07.712296 5659 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:21:07.712713 5659 webserver.cc:460] Webserver started at http://127.4.59.2:46449/ using document root <none> and password file <none>
I20260620 09:21:07.712942 5659 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:07.713007 5659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:07.713197 5659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:07.714174 5659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "ec80fac3256a42f0a14a3c8efbd92a36"
format_stamp: "Formatted at 2026-06-20 09:21:07 on dist-test-slave-414w"
server_key: "7055449b62f7c0af6eaed4491b93d259"
server_key_iv: "0183682baafed2d1176329b7def442ba"
server_key_version: "encryptionkey@0"
I20260620 09:21:07.714519 5659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "ec80fac3256a42f0a14a3c8efbd92a36"
format_stamp: "Formatted at 2026-06-20 09:21:07 on dist-test-slave-414w"
server_key: "7055449b62f7c0af6eaed4491b93d259"
server_key_iv: "0183682baafed2d1176329b7def442ba"
server_key_version: "encryptionkey@0"
I20260620 09:21:07.715747 5659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 09:21:07.716403 5674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:07.716572 5659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:21:07.716678 5659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "ec80fac3256a42f0a14a3c8efbd92a36"
format_stamp: "Formatted at 2026-06-20 09:21:07 on dist-test-slave-414w"
server_key: "7055449b62f7c0af6eaed4491b93d259"
server_key_iv: "0183682baafed2d1176329b7def442ba"
server_key_version: "encryptionkey@0"
I20260620 09:21:07.716763 5659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:07.734726 5659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:07.736385 5659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:07.736534 5659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:07.736979 5659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:07.737035 5659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:07.737097 5659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:07.737126 5659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 09:21:07 dist-test-slave-414w krb5kdc[5402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947267, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:07.744203 5659 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:36283
I20260620 09:21:07.744241 5789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:36283 every 8 connection(s)
I20260620 09:21:07.744560 5659 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:21:07.746685 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.737550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:52911 (local address 127.4.59.62:33003)
0620 09:21:07.737598 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:07.737601 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:07.737816 (+ 215us) server_negotiation.cc:407] Connection header received
0620 09:21:07.738311 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:07.738313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:07.738326 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:07.738387 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:07.739163 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.739992 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.741172 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.741384 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.741486 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.743725 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:07.743727 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:07.743727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:07.743735 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:07.745365 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:07.745714 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:07.745714 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:07.745715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:07.745763 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:07.746044 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:07.746045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:07.746045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:07.746173 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:07.746245 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:07.746522 (+ 277us) server_negotiation.cc:299] Negotiation successful
0620 09:21:07.746593 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 09:21:07.746870 5683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.737525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:33003 (local address 127.4.59.2:52911)
0620 09:21:07.737699 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:07.737731 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:07.738205 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:07.738453 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:07.738457 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:07.738799 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:07.739055 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.739061 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.740096 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.740097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.741051 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.741053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.741511 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.741513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.741655 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.742142 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:07.742151 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:07.743628 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:07.745443 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:07.745447 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:07.745449 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:07.745652 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:07.745840 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:07.745841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:07.745842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:07.745970 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:07.746269 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:07.746274 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:07.746396 (+ 122us) client_negotiation.cc:769] Sending connection context
0620 09:21:07.746499 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 09:21:07.746644 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260620 09:21:07.747318 5790 heartbeater.cc:347] Connected to a master server at 127.4.59.62:33003
I20260620 09:21:07.747393 5790 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:07.747558 5790 heartbeater.cc:511] Master 127.4.59.62:33003 requested a full tablet report, sending...
I20260620 09:21:07.747913 5447 ts_manager.cc:194] Registered new tserver with Master: ec80fac3256a42f0a14a3c8efbd92a36 (127.4.59.2:36283)
I20260620 09:21:07.748394 5447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:52911
I20260620 09:21:07.751791 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 5659
I20260620 09:21:07.751895 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:21:07.752069 4332 external_mini_cluster.cc:1428] Setting key 5a7f6eb148ddea854484fe6331b9f873
I20260620 09:21:07.753641 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.748303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:54755 (local address 127.4.59.62:33003)
0620 09:21:07.748337 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:07.748339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:07.748472 (+ 133us) server_negotiation.cc:407] Connection header received
0620 09:21:07.748524 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:07.748524 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:07.748533 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:07.748559 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:07.748958 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.749456 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.750076 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.750246 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.750292 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.751103 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:07.751104 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:07.751104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:07.751108 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:07.752434 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:07.752810 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:07.752811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:07.752811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:07.752871 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:07.753064 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:07.753065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:07.753066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:07.753266 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:07.753343 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:07.753487 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 09:21:07.753556 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:21:07.753657 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.748234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:33003 (local address 127.4.59.2:54755)
0620 09:21:07.748393 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:07.748404 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:07.748465 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:07.748615 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:07.748616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:07.748817 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:07.748885 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.748887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.749534 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.749535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.750011 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.750012 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.750304 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.750305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.750362 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.750775 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:07.750779 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:07.751036 (+ 257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:07.752505 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:07.752506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:07.752506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:07.752712 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:07.752936 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:07.752937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:07.752938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:07.752993 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:07.753335 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:07.753336 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:07.753377 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 09:21:07.753464 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 09:21:07.753556 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:21:07.777264 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:33003
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:07.860495 5799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:07.860697 5799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:07.860720 5799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:07.862010 5799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:07.862052 5799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:07.862074 5799 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 09:21:07.862092 5799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:21:07.863492 5799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.59.62:33003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:07.863801 5799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:07.864053 5799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:07.868860 5804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:07.869050 5799 server_base.cc:1034] running on GCE node
W20260620 09:21:07.869067 5805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:07.868948 5807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:07.869386 5799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:07.869632 5799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:07.870818 5799 hybrid_clock.cc:648] HybridClock initialized: now 1781947267870803 us; error 44 us; skew 500 ppm
Jun 20 09:21:07 dist-test-slave-414w krb5kdc[5402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947267, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:07.873288 5799 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:21:07.873696 5799 webserver.cc:460] Webserver started at http://127.4.59.3:35775/ using document root <none> and password file <none>
I20260620 09:21:07.873874 5799 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:07.873914 5799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:07.873998 5799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:07.874933 5799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "9f5fba8480ef4692a1e8a91f9c9d904b"
format_stamp: "Formatted at 2026-06-20 09:21:07 on dist-test-slave-414w"
server_key: "5f7a1c081215376f9da4c384f148771a"
server_key_iv: "eb298dda9ea16e73326dd17a5ee75330"
server_key_version: "encryptionkey@0"
I20260620 09:21:07.875211 5799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "9f5fba8480ef4692a1e8a91f9c9d904b"
format_stamp: "Formatted at 2026-06-20 09:21:07 on dist-test-slave-414w"
server_key: "5f7a1c081215376f9da4c384f148771a"
server_key_iv: "eb298dda9ea16e73326dd17a5ee75330"
server_key_version: "encryptionkey@0"
I20260620 09:21:07.876693 5799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:21:07.877502 5814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:07.877671 5799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:07.877756 5799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "9f5fba8480ef4692a1e8a91f9c9d904b"
format_stamp: "Formatted at 2026-06-20 09:21:07 on dist-test-slave-414w"
server_key: "5f7a1c081215376f9da4c384f148771a"
server_key_iv: "eb298dda9ea16e73326dd17a5ee75330"
server_key_version: "encryptionkey@0"
I20260620 09:21:07.877817 5799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:07.892293 5799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:07.894230 5799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:07.894348 5799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:07.894769 5799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:07.894806 5799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:07.894860 5799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:07.894887 5799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 09:21:07 dist-test-slave-414w krb5kdc[5402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947267, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:07.902256 5799 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:38667
I20260620 09:21:07.902333 5929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:38667 every 8 connection(s)
I20260620 09:21:07.902649 5799 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:21:07.904937 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.895285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:36385 (local address 127.4.59.62:33003)
0620 09:21:07.895364 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:07.895367 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:07.895502 (+ 135us) server_negotiation.cc:407] Connection header received
0620 09:21:07.896004 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:07.896006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:07.896019 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:07.896107 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:07.896908 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.897868 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.898901 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.899155 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.899238 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.902009 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:07.902010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:07.902011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:07.902018 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:07.903582 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:07.903994 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:07.903996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:07.903997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:07.904052 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:07.904319 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:07.904320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:07.904320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:07.904434 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:07.904508 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:07.904772 (+ 264us) server_negotiation.cc:299] Negotiation successful
0620 09:21:07.904848 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 09:21:07.905110 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.895200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:33003 (local address 127.4.59.3:36385)
0620 09:21:07.895376 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:07.895413 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:07.895902 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:07.896196 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:07.896200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:07.896537 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:07.896798 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.896804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.897986 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.897987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.898792 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.898794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.899264 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.899265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.899402 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.899909 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:07.899920 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:07.901901 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:07.903679 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:07.903685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:07.903687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:07.903901 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:07.904126 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:07.904127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:07.904127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:07.904242 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:07.904499 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:07.904505 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:07.904637 (+ 132us) client_negotiation.cc:769] Sending connection context
0620 09:21:07.904744 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 09:21:07.904890 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260620 09:21:07.905561 5930 heartbeater.cc:347] Connected to a master server at 127.4.59.62:33003
I20260620 09:21:07.905643 5930 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:07.905825 5930 heartbeater.cc:511] Master 127.4.59.62:33003 requested a full tablet report, sending...
I20260620 09:21:07.906185 5447 ts_manager.cc:194] Registered new tserver with Master: 9f5fba8480ef4692a1e8a91f9c9d904b (127.4.59.3:38667)
I20260620 09:21:07.906584 5447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:36385
I20260620 09:21:07.911653 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 5799
I20260620 09:21:07.911715 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:21:07.911919 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.906537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:33003 (local address 127.4.59.3:41637)
0620 09:21:07.906646 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:07.906656 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:07.906726 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:07.906860 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:07.906860 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:07.907031 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:07.907087 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.907088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.907715 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.907716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.908231 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.908239 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.908610 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.908610 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.908684 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.909111 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:07.909116 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:07.909417 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:07.910968 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:07.910969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:07.910970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:07.911133 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:07.911331 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:07.911332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:07.911333 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:07.911388 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:07.911633 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:07.911634 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:07.911673 (+ 39us) client_negotiation.cc:769] Sending connection context
0620 09:21:07.911745 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 09:21:07.911841 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":60,"threads_started":1}
I20260620 09:21:07.911931 4332 external_mini_cluster.cc:1428] Setting key 75503622383f1d45b78ee9aedb625d30
I20260620 09:21:07.911971 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.906598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:41637 (local address 127.4.59.62:33003)
0620 09:21:07.906644 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:07.906646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:07.906704 (+ 58us) server_negotiation.cc:407] Connection header received
0620 09:21:07.906786 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:07.906786 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:07.906795 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:07.906856 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:07.907164 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.907622 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.908321 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.908525 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.908607 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.909522 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:07.909523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:07.909524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:07.909530 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:07.910869 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:07.911205 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:07.911206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:07.911206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:07.911257 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:07.911460 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:07.911461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:07.911461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:07.911567 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:07.911642 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:07.911801 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 09:21:07.911879 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 09:21:07.912776 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 09:21:07 dist-test-slave-414w krb5kdc[5402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947267, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 09:21:07 dist-test-slave-414w krb5kdc[5402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947267, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:07.935665 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.928406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59266 (local address 127.4.59.62:33003)
0620 09:21:07.928472 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:07.928474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:07.928484 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:21:07.928532 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:07.928533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:07.928549 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:07.928627 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:07.929316 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.929807 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.930544 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.930734 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.930819 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.932926 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:07.932927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:07.932927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:07.932935 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:07.934500 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:07.934876 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:07.934877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:07.934878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:07.934935 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:07.935149 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:07.935150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:07.935150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:07.935269 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:07.935377 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:07.935486 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 09:21:07.935562 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 09:21:07.937278 5447 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:21:07.938748 5447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:21:07.945366 5948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.940971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:36283 (local address 127.0.0.1:45492)
0620 09:21:07.941109 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:07.941171 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:07.941236 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:07.941538 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:07.941541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:07.941543 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:07.941604 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.941605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.942843 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.942844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.944144 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.944147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.945078 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.945079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.945134 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.945202 (+ 68us) client_negotiation.cc:769] Sending connection context
0620 09:21:07.945274 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 09:21:07.945311 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":83,"thread_start_us":63,"threads_started":1}
I20260620 09:21:07.945578 5949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.941189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45492 (local address 127.4.59.2:36283)
0620 09:21:07.941385 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:07.941390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:07.941400 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:21:07.941438 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:07.941441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:07.941457 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:07.941535 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:07.941673 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.942749 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.944242 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.944994 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.945080 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.945167 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:07.945309 (+ 142us) server_negotiation.cc:299] Negotiation successful
0620 09:21:07.945356 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":85,"threads_started":1}
I20260620 09:21:07.945868 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.941237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:39233 (local address 127.0.0.1:57496)
0620 09:21:07.941427 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:07.941439 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:07.941497 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:07.941818 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:07.941819 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:07.941821 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:07.941863 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.941865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.943024 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.943024 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.944404 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.944406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.945425 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.945426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.945491 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.945496 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 09:21:07.945539 (+ 43us) client_negotiation.cc:240] Negotiation successful
0620 09:21:07.945576 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":66,"thread_start_us":88,"threads_started":1}
I20260620 09:21:07.946341 5724 tablet_service.cc:1467] Processing CreateTablet for tablet 2f9373a57343424eb5c697ff762eee5b (DEFAULT_TABLE table=test-table [id=0018dc76f50f4e4f979ea6beeace030d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:07.946760 5724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f9373a57343424eb5c697ff762eee5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:07.946863 5952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.941480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57496 (local address 127.4.59.1:39233)
0620 09:21:07.941649 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:07.941654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:07.941665 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:21:07.941707 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:07.941710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:07.941726 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:07.941814 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:07.941934 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.942918 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.944483 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.945339 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.945417 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.946674 (+ 1257us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:07.946703 (+ 29us) server_negotiation.cc:299] Negotiation successful
0620 09:21:07.946749 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260620 09:21:07.947190 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.941244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47046 (local address 127.4.59.3:38667)
0620 09:21:07.941686 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:07.941690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:07.941701 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:21:07.941737 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:07.941739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:07.941757 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:07.942078 (+ 321us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:07.942300 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.943163 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.945634 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.946452 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.946560 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.946620 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:07.947016 (+ 396us) server_negotiation.cc:299] Negotiation successful
0620 09:21:07.947062 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":83,"threads_started":1}
I20260620 09:21:07.947410 5950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.940974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:38667 (local address 127.0.0.1:47046)
0620 09:21:07.941360 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:07.941369 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:07.941409 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:07.942162 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:07.942163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:07.942165 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:07.942215 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.942216 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.943255 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.943256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.944400 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.944403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.946605 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.946606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.946698 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.946703 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 09:21:07.947290 (+ 587us) client_negotiation.cc:240] Negotiation successful
0620 09:21:07.947340 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":84,"threads_started":1}
I20260620 09:21:07.947518 5584 tablet_service.cc:1467] Processing CreateTablet for tablet 2f9373a57343424eb5c697ff762eee5b (DEFAULT_TABLE table=test-table [id=0018dc76f50f4e4f979ea6beeace030d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:07.947865 5584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f9373a57343424eb5c697ff762eee5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:07.948073 5864 tablet_service.cc:1467] Processing CreateTablet for tablet 2f9373a57343424eb5c697ff762eee5b (DEFAULT_TABLE table=test-table [id=0018dc76f50f4e4f979ea6beeace030d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:07.948453 5864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f9373a57343424eb5c697ff762eee5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:07.950951 5953 tablet_bootstrap.cc:492] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36: Bootstrap starting.
I20260620 09:21:07.951344 5954 tablet_bootstrap.cc:492] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3: Bootstrap starting.
I20260620 09:21:07.951606 5953 tablet_bootstrap.cc:654] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:07.951902 5953 log.cc:824] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:07.951928 5955 tablet_bootstrap.cc:492] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b: Bootstrap starting.
I20260620 09:21:07.952026 5954 tablet_bootstrap.cc:654] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:07.952359 5954 log.cc:824] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:07.952858 5955 tablet_bootstrap.cc:654] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:07.953104 5955 log.cc:824] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:07.953397 5954 tablet_bootstrap.cc:492] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3: No bootstrap required, opened a new log
I20260620 09:21:07.953480 5954 ts_tablet_manager.cc:1397] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:21:07.953678 5953 tablet_bootstrap.cc:492] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36: No bootstrap required, opened a new log
I20260620 09:21:07.953776 5953 ts_tablet_manager.cc:1397] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 09:21:07.954052 5955 tablet_bootstrap.cc:492] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b: No bootstrap required, opened a new log
I20260620 09:21:07.954119 5955 ts_tablet_manager.cc:1397] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260620 09:21:07.955148 5954 raft_consensus.cc:348] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:07.955312 5954 raft_consensus.cc:374] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:07.955360 5954 raft_consensus.cc:729] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 176c1ee9098d4fef911112f3b54995b3, State: Initialized, Role: FOLLOWER
I20260620 09:21:07.955514 5954 consensus_queue.cc:260] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:07.955806 5954 ts_tablet_manager.cc:1428] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:21:07.955741 5953 raft_consensus.cc:348] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:07.955888 5953 raft_consensus.cc:374] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:07.955920 5953 raft_consensus.cc:729] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec80fac3256a42f0a14a3c8efbd92a36, State: Initialized, Role: FOLLOWER
I20260620 09:21:07.955935 5650 heartbeater.cc:503] Master 127.4.59.62:33003 was elected leader, sending a full tablet report...
I20260620 09:21:07.956019 5953 consensus_queue.cc:260] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:07.955998 5955 raft_consensus.cc:348] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:07.956127 5955 raft_consensus.cc:374] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:07.956166 5955 raft_consensus.cc:729] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f5fba8480ef4692a1e8a91f9c9d904b, State: Initialized, Role: FOLLOWER
I20260620 09:21:07.956267 5953 ts_tablet_manager.cc:1428] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 09:21:07.956308 5955 consensus_queue.cc:260] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:07.956383 5790 heartbeater.cc:503] Master 127.4.59.62:33003 was elected leader, sending a full tablet report...
I20260620 09:21:07.956563 5955 ts_tablet_manager.cc:1428] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 09:21:07.956676 5930 heartbeater.cc:503] Master 127.4.59.62:33003 was elected leader, sending a full tablet report...
I20260620 09:21:07.957512 5962 raft_consensus.cc:482] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:21:07.957612 5962 raft_consensus.cc:504] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:07.957968 5962 leader_election.cc:288] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec80fac3256a42f0a14a3c8efbd92a36 (127.4.59.2:36283), 9f5fba8480ef4692a1e8a91f9c9d904b (127.4.59.3:38667)
I20260620 09:21:07.960949 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.958310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:40557 (local address 127.4.59.3:38667)
0620 09:21:07.958352 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:07.958354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:07.958362 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:21:07.958382 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:07.958383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:07.958392 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:07.958446 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:07.958588 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.959131 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.960216 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.960589 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.960684 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.960694 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:07.960856 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 09:21:07.960891 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 09:21:07.960974 5963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.958135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:38667 (local address 127.4.59.1:40557)
0620 09:21:07.958275 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:07.958285 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:07.958323 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:07.958450 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:07.958450 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:07.958452 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:07.958512 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.958513 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.959228 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.959229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.960116 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.960120 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.960693 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.960694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.960765 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.960776 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 09:21:07.960848 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 09:21:07.960886 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":30,"thread_start_us":83,"threads_started":1}
I20260620 09:21:07.961078 5949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.958139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:47003 (local address 127.4.59.2:36283)
0620 09:21:07.958184 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:07.958187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:07.958198 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:21:07.958301 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:07.958302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:07.958313 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:07.958371 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:07.958576 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.959142 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.960290 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.960682 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.960780 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.960790 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:07.960974 (+ 184us) server_negotiation.cc:299] Negotiation successful
0620 09:21:07.961011 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 09:21:07.961082 5543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.958101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:36283 (local address 127.4.59.1:47003)
0620 09:21:07.958153 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:07.958163 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:07.958210 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:07.958376 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:07.958377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:07.958379 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:07.958441 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.958442 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.959236 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.959237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.960222 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:07.960225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.960783 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.960799 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:07.960894 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.960897 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:21:07.960970 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 09:21:07.961011 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":24}
I20260620 09:21:07.961311 5884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f9373a57343424eb5c697ff762eee5b" candidate_uuid: "176c1ee9098d4fef911112f3b54995b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" is_pre_election: true
I20260620 09:21:07.961373 5744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f9373a57343424eb5c697ff762eee5b" candidate_uuid: "176c1ee9098d4fef911112f3b54995b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" is_pre_election: true
I20260620 09:21:07.961438 5884 raft_consensus.cc:2454] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 176c1ee9098d4fef911112f3b54995b3 in term 0.
I20260620 09:21:07.961477 5744 raft_consensus.cc:2454] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 176c1ee9098d4fef911112f3b54995b3 in term 0.
I20260620 09:21:07.961695 5538 leader_election.cc:302] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 176c1ee9098d4fef911112f3b54995b3, 9f5fba8480ef4692a1e8a91f9c9d904b, ec80fac3256a42f0a14a3c8efbd92a36; no voters:
I20260620 09:21:07.961786 5962 raft_consensus.cc:2781] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:21:07.961838 5962 raft_consensus.cc:482] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:21:07.961859 5962 raft_consensus.cc:3037] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:07.962679 5962 raft_consensus.cc:504] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:07.962821 5962 leader_election.cc:288] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [CANDIDATE]: Term 1 election: Requested vote from peers ec80fac3256a42f0a14a3c8efbd92a36 (127.4.59.2:36283), 9f5fba8480ef4692a1e8a91f9c9d904b (127.4.59.3:38667)
I20260620 09:21:07.962942 5744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f9373a57343424eb5c697ff762eee5b" candidate_uuid: "176c1ee9098d4fef911112f3b54995b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec80fac3256a42f0a14a3c8efbd92a36"
I20260620 09:21:07.963023 5744 raft_consensus.cc:3037] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:07.963027 5884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f9373a57343424eb5c697ff762eee5b" candidate_uuid: "176c1ee9098d4fef911112f3b54995b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b"
I20260620 09:21:07.963130 5884 raft_consensus.cc:3037] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:07.963701 5744 raft_consensus.cc:2454] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 176c1ee9098d4fef911112f3b54995b3 in term 1.
I20260620 09:21:07.963863 5884 raft_consensus.cc:2454] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 176c1ee9098d4fef911112f3b54995b3 in term 1.
I20260620 09:21:07.963887 5537 leader_election.cc:302] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 176c1ee9098d4fef911112f3b54995b3, ec80fac3256a42f0a14a3c8efbd92a36; no voters:
I20260620 09:21:07.964061 5962 raft_consensus.cc:2781] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:07.964112 5962 raft_consensus.cc:686] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [term 1 LEADER]: Becoming Leader. State: Replica: 176c1ee9098d4fef911112f3b54995b3, State: Running, Role: LEADER
I20260620 09:21:07.964184 5962 consensus_queue.cc:237] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:07.964943 5447 catalog_manager.cc:5495] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 176c1ee9098d4fef911112f3b54995b3 (127.4.59.1). New cstate: current_term: 1 leader_uuid: "176c1ee9098d4fef911112f3b54995b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } health_report { overall_health: HEALTHY } } }
I20260620 09:21:07.977272 5952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:07.974703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57506 (local address 127.4.59.1:39233)
0620 09:21:07.974781 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:07.974783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:07.974793 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:21:07.974883 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:07.974884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:07.974901 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:07.974964 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:21:07.975109 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.975576 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.976377 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:07.976578 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:07.976659 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:07.976893 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:21:07.976986 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:21:07.977073 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:07.977126 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 09:21:07.977168 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 09:21:07.979069 5744 raft_consensus.cc:1261] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36 [term 1 FOLLOWER]: Refusing update from remote peer 176c1ee9098d4fef911112f3b54995b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:07.979259 5884 raft_consensus.cc:1261] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b [term 1 FOLLOWER]: Refusing update from remote peer 176c1ee9098d4fef911112f3b54995b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:07.979370 5962 consensus_queue.cc:1035] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:07.979534 5959 consensus_queue.cc:1035] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:07.981866 5967 mvcc.cc:205] Tried to move back new op lower bound from 7298856009639243776 to 7298856009581600768. Current Snapshot: MvccSnapshot[applied={T|T < 7298856009639243776}]
I20260620 09:21:07.985353 5447 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:07.985463 5447 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:07.986166 5447 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da: Sending DeleteTablet for 3 replicas of tablet 2f9373a57343424eb5c697ff762eee5b
I20260620 09:21:07.986510 5584 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f9373a57343424eb5c697ff762eee5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:07 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:57496
I20260620 09:21:07.986527 5864 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f9373a57343424eb5c697ff762eee5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:07 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:47046
I20260620 09:21:07.986742 5447 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:21:07.986903 5447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:21:07.986717 5724 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f9373a57343424eb5c697ff762eee5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:07 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:45492
I20260620 09:21:07.987361 5977 tablet_replica.cc:330] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b: stopping tablet replica
I20260620 09:21:07.987469 5977 raft_consensus.cc:2229] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:07.987579 5977 raft_consensus.cc:2258] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:07.988083 5864 tablet_service.cc:1467] Processing CreateTablet for tablet 734ebf7d1e914a798f3861351690e49c (DEFAULT_TABLE table=test-table [id=6b17f54141f0497b87d809423089763c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:07.988106 5724 tablet_service.cc:1467] Processing CreateTablet for tablet 734ebf7d1e914a798f3861351690e49c (DEFAULT_TABLE table=test-table [id=6b17f54141f0497b87d809423089763c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:07.988116 5584 tablet_service.cc:1467] Processing CreateTablet for tablet 734ebf7d1e914a798f3861351690e49c (DEFAULT_TABLE table=test-table [id=6b17f54141f0497b87d809423089763c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:07.988204 5584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 734ebf7d1e914a798f3861351690e49c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:07.988204 5864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 734ebf7d1e914a798f3861351690e49c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:07.988256 5977 ts_tablet_manager.cc:1905] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:07.989528 5955 tablet_bootstrap.cc:492] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b: Bootstrap starting.
I20260620 09:21:07.989548 5954 tablet_bootstrap.cc:492] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3: Bootstrap starting.
I20260620 09:21:07.989926 5955 tablet_bootstrap.cc:654] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:07.989935 5954 tablet_bootstrap.cc:654] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:07.990581 5955 tablet_bootstrap.cc:492] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b: No bootstrap required, opened a new log
I20260620 09:21:07.990628 5955 ts_tablet_manager.cc:1397] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 09:21:07.990626 5724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 734ebf7d1e914a798f3861351690e49c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:07.990777 5955 raft_consensus.cc:348] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } }
I20260620 09:21:07.990818 5978 tablet_replica.cc:330] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36: stopping tablet replica
I20260620 09:21:07.990859 5955 raft_consensus.cc:374] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:07.990878 5955 raft_consensus.cc:729] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f5fba8480ef4692a1e8a91f9c9d904b, State: Initialized, Role: FOLLOWER
I20260620 09:21:07.990916 5955 consensus_queue.cc:260] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } }
I20260620 09:21:07.990943 5978 raft_consensus.cc:2229] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:07.990991 5955 ts_tablet_manager.cc:1428] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:07.991065 5978 raft_consensus.cc:2258] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:07.991477 5978 ts_tablet_manager.cc:1905] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:07.992053 5953 tablet_bootstrap.cc:492] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36: Bootstrap starting.
I20260620 09:21:07.992437 5953 tablet_bootstrap.cc:654] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:07.992944 5978 ts_tablet_manager.cc:1918] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:21:07.993026 5953 tablet_bootstrap.cc:492] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36: No bootstrap required, opened a new log
I20260620 09:21:07.993041 5978 log.cc:1196] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/2f9373a57343424eb5c697ff762eee5b
I20260620 09:21:07.993072 5953 ts_tablet_manager.cc:1397] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:07.993213 5953 raft_consensus.cc:348] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } }
I20260620 09:21:07.993299 5953 raft_consensus.cc:374] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:07.993322 5953 raft_consensus.cc:729] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec80fac3256a42f0a14a3c8efbd92a36, State: Initialized, Role: FOLLOWER
I20260620 09:21:07.993335 5978 ts_tablet_manager.cc:1939] T 2f9373a57343424eb5c697ff762eee5b P ec80fac3256a42f0a14a3c8efbd92a36: Deleting consensus metadata
I20260620 09:21:07.993364 5953 consensus_queue.cc:260] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } }
I20260620 09:21:07.993450 5953 ts_tablet_manager.cc:1428] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260620 09:21:07.993724 5433 catalog_manager.cc:4841] TS ec80fac3256a42f0a14a3c8efbd92a36 (127.4.59.2:36283): tablet 2f9373a57343424eb5c697ff762eee5b (table test-table [id=0018dc76f50f4e4f979ea6beeace030d]) successfully deleted
I20260620 09:21:07.992843 5977 ts_tablet_manager.cc:1918] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:21:07.993923 5977 log.cc:1196] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/2f9373a57343424eb5c697ff762eee5b
I20260620 09:21:07.994194 5977 ts_tablet_manager.cc:1939] T 2f9373a57343424eb5c697ff762eee5b P 9f5fba8480ef4692a1e8a91f9c9d904b: Deleting consensus metadata
I20260620 09:21:07.994562 5434 catalog_manager.cc:4841] TS 9f5fba8480ef4692a1e8a91f9c9d904b (127.4.59.3:38667): tablet 2f9373a57343424eb5c697ff762eee5b (table test-table [id=0018dc76f50f4e4f979ea6beeace030d]) successfully deleted
I20260620 09:21:07.994801 5954 tablet_bootstrap.cc:492] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3: No bootstrap required, opened a new log
I20260620 09:21:07.994868 5954 ts_tablet_manager.cc:1397] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260620 09:21:07.995052 5954 raft_consensus.cc:348] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } }
W20260620 09:21:07.995103 5791 tablet.cc:2369] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:21:07.995132 5954 raft_consensus.cc:374] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:07.995154 5954 raft_consensus.cc:729] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 176c1ee9098d4fef911112f3b54995b3, State: Initialized, Role: FOLLOWER
I20260620 09:21:07.995195 5954 consensus_queue.cc:260] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } }
I20260620 09:21:07.995294 5954 ts_tablet_manager.cc:1428] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:07.997260 5976 tablet_replica.cc:330] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3: stopping tablet replica
I20260620 09:21:07.997370 5976 raft_consensus.cc:2229] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:21:07.997498 5976 raft_consensus.cc:2258] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:07.997879 5976 ts_tablet_manager.cc:1905] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:07.999297 5976 ts_tablet_manager.cc:1918] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:21:07.999400 5976 log.cc:1196] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/wal/wals/2f9373a57343424eb5c697ff762eee5b
I20260620 09:21:07.999689 5976 ts_tablet_manager.cc:1939] T 2f9373a57343424eb5c697ff762eee5b P 176c1ee9098d4fef911112f3b54995b3: Deleting consensus metadata
I20260620 09:21:08.000092 5433 catalog_manager.cc:4841] TS 176c1ee9098d4fef911112f3b54995b3 (127.4.59.1:39233): tablet 2f9373a57343424eb5c697ff762eee5b (table test-table [id=0018dc76f50f4e4f979ea6beeace030d]) successfully deleted
W20260620 09:21:08.087286 5651 tablet.cc:2369] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 09:21:08.153242 5931 tablet.cc:2369] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:21:08.168591 5960 raft_consensus.cc:482] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:21:08.168705 5960 raft_consensus.cc:504] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } }
I20260620 09:21:08.168993 5960 leader_election.cc:288] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 176c1ee9098d4fef911112f3b54995b3 (127.4.59.1:39233), 9f5fba8480ef4692a1e8a91f9c9d904b (127.4.59.3:38667)
I20260620 09:21:08.172995 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:08.169317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:52833 (local address 127.4.59.3:38667)
0620 09:21:08.169364 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:08.169366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:08.169397 (+ 31us) server_negotiation.cc:407] Connection header received
0620 09:21:08.169496 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:08.169516 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:08.169528 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:08.169602 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:08.169808 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.170556 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.172058 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.172599 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.172695 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:08.172706 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:08.172874 (+ 168us) server_negotiation.cc:299] Negotiation successful
0620 09:21:08.172915 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 09:21:08.172995 5952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:08.169344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:56561 (local address 127.4.59.1:39233)
0620 09:21:08.169394 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:08.169396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:08.169534 (+ 138us) server_negotiation.cc:407] Connection header received
0620 09:21:08.169579 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:08.169580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:08.169589 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:08.169638 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:08.169813 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.170545 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.172095 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.172611 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.172694 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:08.172706 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:08.172878 (+ 172us) server_negotiation.cc:299] Negotiation successful
0620 09:21:08.172916 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 09:21:08.173032 5683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:08.169256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:38667 (local address 127.4.59.2:52833)
0620 09:21:08.169329 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:08.169341 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:08.169416 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:08.169612 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:08.169629 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:08.169630 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:08.169720 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:08.169722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.170641 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.170642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:08.171955 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:08.171959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.172699 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.172700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:08.172785 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:08.172789 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:21:08.172865 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 09:21:08.172915 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":55}
I20260620 09:21:08.173056 5982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:08.169256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:39233 (local address 127.4.59.2:56561)
0620 09:21:08.169451 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:08.169461 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:08.169522 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:08.169652 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:08.169653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:08.169654 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:08.169729 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:08.169730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.170641 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.170642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:08.171956 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:08.171959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.172718 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.172719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:08.172799 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:08.172803 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:21:08.172871 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 09:21:08.172916 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":158,"thread_start_us":58,"threads_started":1}
I20260620 09:21:08.173344 5604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "734ebf7d1e914a798f3861351690e49c" candidate_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "176c1ee9098d4fef911112f3b54995b3" is_pre_election: true
I20260620 09:21:08.173373 5884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "734ebf7d1e914a798f3861351690e49c" candidate_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" is_pre_election: true
I20260620 09:21:08.173463 5884 raft_consensus.cc:2454] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec80fac3256a42f0a14a3c8efbd92a36 in term 0.
I20260620 09:21:08.173480 5604 raft_consensus.cc:2454] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec80fac3256a42f0a14a3c8efbd92a36 in term 0.
I20260620 09:21:08.173647 5678 leader_election.cc:302] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f5fba8480ef4692a1e8a91f9c9d904b, ec80fac3256a42f0a14a3c8efbd92a36; no voters:
I20260620 09:21:08.173791 5960 raft_consensus.cc:2781] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:21:08.173835 5960 raft_consensus.cc:482] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:21:08.173856 5960 raft_consensus.cc:3037] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:08.174523 5960 raft_consensus.cc:504] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } }
I20260620 09:21:08.174664 5960 leader_election.cc:288] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [CANDIDATE]: Term 1 election: Requested vote from peers 176c1ee9098d4fef911112f3b54995b3 (127.4.59.1:39233), 9f5fba8480ef4692a1e8a91f9c9d904b (127.4.59.3:38667)
I20260620 09:21:08.174803 5884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "734ebf7d1e914a798f3861351690e49c" candidate_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b"
I20260620 09:21:08.174875 5884 raft_consensus.cc:3037] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:08.174808 5604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "734ebf7d1e914a798f3861351690e49c" candidate_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "176c1ee9098d4fef911112f3b54995b3"
I20260620 09:21:08.174907 5604 raft_consensus.cc:3037] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:08.175498 5884 raft_consensus.cc:2454] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec80fac3256a42f0a14a3c8efbd92a36 in term 1.
I20260620 09:21:08.175596 5604 raft_consensus.cc:2454] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec80fac3256a42f0a14a3c8efbd92a36 in term 1.
I20260620 09:21:08.175655 5678 leader_election.cc:302] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f5fba8480ef4692a1e8a91f9c9d904b, ec80fac3256a42f0a14a3c8efbd92a36; no voters:
I20260620 09:21:08.175794 5960 raft_consensus.cc:2781] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:08.175941 5960 raft_consensus.cc:686] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [term 1 LEADER]: Becoming Leader. State: Replica: ec80fac3256a42f0a14a3c8efbd92a36, State: Running, Role: LEADER
I20260620 09:21:08.176035 5960 consensus_queue.cc:237] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } }
I20260620 09:21:08.177063 5447 catalog_manager.cc:5495] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec80fac3256a42f0a14a3c8efbd92a36 (127.4.59.2). New cstate: current_term: 1 leader_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } health_report { overall_health: UNKNOWN } } }
Jun 20 09:21:08 dist-test-slave-414w krb5kdc[5402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947267, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:08.244783 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:08.237953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59282 (local address 127.4.59.62:33003)
0620 09:21:08.238016 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:08.238018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:08.238123 (+ 105us) server_negotiation.cc:407] Connection header received
0620 09:21:08.238214 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:08.238215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:08.238226 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:08.238298 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:08.238743 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.239234 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.240116 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.240310 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.240407 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:08.242626 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:08.242627 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:08.242628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:08.242634 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:08.243839 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:08.244182 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:08.244183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:08.244183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:08.244230 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:08.244397 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:08.244398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:08.244398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:08.244491 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:08.244539 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:08.244657 (+ 118us) server_negotiation.cc:299] Negotiation successful
0620 09:21:08.244718 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 09:21:08.244895 5990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:08.237902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:33003 (local address 127.0.0.1:59282)
0620 09:21:08.238053 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:08.238062 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:08.238143 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:08.238303 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:08.238303 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:08.238609 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:08.238654 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:08.238656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.239345 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.239346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:08.240022 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:08.240024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.240421 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.240422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:08.240478 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:08.241047 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:08.241052 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:08.242540 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:08.243914 (+ 1374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:08.243920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:08.243923 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:08.244118 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:08.244289 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:08.244290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:08.244290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:08.244345 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:08.244543 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:08.244548 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:08.244592 (+ 44us) client_negotiation.cc:769] Sending connection context
0620 09:21:08.244643 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 09:21:08.244768 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":42,"thread_start_us":72,"threads_started":1}
I20260620 09:21:08.246779 5447 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:59282:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 09:21:08.247052 5447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:21:08.248520 5724 tablet_service.cc:1467] Processing CreateTablet for tablet 2d07051c4b234dbbaad96f9cdde56d76 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f583a5713eac41dba7e7090c4626c6fb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 09:21:08.248543 5584 tablet_service.cc:1467] Processing CreateTablet for tablet 2d07051c4b234dbbaad96f9cdde56d76 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f583a5713eac41dba7e7090c4626c6fb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 09:21:08.248534 5864 tablet_service.cc:1467] Processing CreateTablet for tablet 2d07051c4b234dbbaad96f9cdde56d76 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f583a5713eac41dba7e7090c4626c6fb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 09:21:08.248678 5584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d07051c4b234dbbaad96f9cdde56d76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:08.248677 5864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d07051c4b234dbbaad96f9cdde56d76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:08.248679 5724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d07051c4b234dbbaad96f9cdde56d76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:08.249951 5953 tablet_bootstrap.cc:492] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36: Bootstrap starting.
I20260620 09:21:08.250330 5955 tablet_bootstrap.cc:492] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b: Bootstrap starting.
I20260620 09:21:08.250344 5953 tablet_bootstrap.cc:654] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:08.250764 5955 tablet_bootstrap.cc:654] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:08.251019 5953 tablet_bootstrap.cc:492] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36: No bootstrap required, opened a new log
I20260620 09:21:08.251060 5953 ts_tablet_manager.cc:1397] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 09:21:08.251087 5954 tablet_bootstrap.cc:492] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3: Bootstrap starting.
I20260620 09:21:08.251194 5953 raft_consensus.cc:348] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:08.251266 5953 raft_consensus.cc:374] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:08.251298 5953 raft_consensus.cc:729] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec80fac3256a42f0a14a3c8efbd92a36, State: Initialized, Role: FOLLOWER
I20260620 09:21:08.251350 5953 consensus_queue.cc:260] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:08.251379 5955 tablet_bootstrap.cc:492] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b: No bootstrap required, opened a new log
I20260620 09:21:08.251423 5955 ts_tablet_manager.cc:1397] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 09:21:08.251418 5954 tablet_bootstrap.cc:654] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:08.251459 5953 ts_tablet_manager.cc:1428] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:08.251502 5960 tablet_replica.cc:439] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } }
I20260620 09:21:08.251590 5960 tablet_replica.cc:442] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:21:08.251561 5955 raft_consensus.cc:348] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:08.251622 5955 raft_consensus.cc:374] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:08.251645 5955 raft_consensus.cc:729] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f5fba8480ef4692a1e8a91f9c9d904b, State: Initialized, Role: FOLLOWER
I20260620 09:21:08.251699 5955 consensus_queue.cc:260] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:08.251796 5955 ts_tablet_manager.cc:1428] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260620 09:21:08.251919 5961 tablet_replica.cc:439] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } }
I20260620 09:21:08.251994 5954 tablet_bootstrap.cc:492] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3: No bootstrap required, opened a new log
I20260620 09:21:08.252010 5961 tablet_replica.cc:442] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:21:08.252050 5954 ts_tablet_manager.cc:1397] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:08.252161 5954 raft_consensus.cc:348] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:08.252262 5954 raft_consensus.cc:374] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:08.252285 5954 raft_consensus.cc:729] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 176c1ee9098d4fef911112f3b54995b3, State: Initialized, Role: FOLLOWER
I20260620 09:21:08.252334 5954 consensus_queue.cc:260] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:08.252422 5954 ts_tablet_manager.cc:1428] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:08.252499 5962 tablet_replica.cc:439] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } }
I20260620 09:21:08.252604 5962 tablet_replica.cc:442] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:21:08.422472 5962 raft_consensus.cc:482] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:21:08.422569 5962 raft_consensus.cc:504] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:08.422725 5962 leader_election.cc:288] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec80fac3256a42f0a14a3c8efbd92a36 (127.4.59.2:36283), 9f5fba8480ef4692a1e8a91f9c9d904b (127.4.59.3:38667)
I20260620 09:21:08.422977 5744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d07051c4b234dbbaad96f9cdde56d76" candidate_uuid: "176c1ee9098d4fef911112f3b54995b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" is_pre_election: true
I20260620 09:21:08.422977 5884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d07051c4b234dbbaad96f9cdde56d76" candidate_uuid: "176c1ee9098d4fef911112f3b54995b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" is_pre_election: true
I20260620 09:21:08.423084 5884 raft_consensus.cc:2454] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 176c1ee9098d4fef911112f3b54995b3 in term 0.
I20260620 09:21:08.423084 5744 raft_consensus.cc:2454] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 176c1ee9098d4fef911112f3b54995b3 in term 0.
I20260620 09:21:08.423220 5537 leader_election.cc:302] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 176c1ee9098d4fef911112f3b54995b3, ec80fac3256a42f0a14a3c8efbd92a36; no voters:
I20260620 09:21:08.423348 5962 raft_consensus.cc:2781] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:21:08.423393 5962 raft_consensus.cc:482] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:21:08.423413 5962 raft_consensus.cc:3037] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:08.424015 5962 raft_consensus.cc:504] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:08.424144 5962 leader_election.cc:288] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [CANDIDATE]: Term 1 election: Requested vote from peers ec80fac3256a42f0a14a3c8efbd92a36 (127.4.59.2:36283), 9f5fba8480ef4692a1e8a91f9c9d904b (127.4.59.3:38667)
I20260620 09:21:08.424292 5884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d07051c4b234dbbaad96f9cdde56d76" candidate_uuid: "176c1ee9098d4fef911112f3b54995b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b"
I20260620 09:21:08.424281 5744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d07051c4b234dbbaad96f9cdde56d76" candidate_uuid: "176c1ee9098d4fef911112f3b54995b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec80fac3256a42f0a14a3c8efbd92a36"
I20260620 09:21:08.424376 5744 raft_consensus.cc:3037] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:08.424377 5884 raft_consensus.cc:3037] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:08.424958 5884 raft_consensus.cc:2454] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 176c1ee9098d4fef911112f3b54995b3 in term 1.
I20260620 09:21:08.425025 5744 raft_consensus.cc:2454] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 176c1ee9098d4fef911112f3b54995b3 in term 1.
I20260620 09:21:08.425175 5538 leader_election.cc:302] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 176c1ee9098d4fef911112f3b54995b3, 9f5fba8480ef4692a1e8a91f9c9d904b; no voters:
I20260620 09:21:08.425304 5962 raft_consensus.cc:2781] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:08.425381 5962 raft_consensus.cc:686] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [term 1 LEADER]: Becoming Leader. State: Replica: 176c1ee9098d4fef911112f3b54995b3, State: Running, Role: LEADER
I20260620 09:21:08.425436 5962 consensus_queue.cc:237] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } }
I20260620 09:21:08.425693 5971 tablet_replica.cc:439] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3: TxnStatusTablet state changed. Reason: New leader 176c1ee9098d4fef911112f3b54995b3. Latest consensus state: current_term: 1 leader_uuid: "176c1ee9098d4fef911112f3b54995b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } }
I20260620 09:21:08.425783 5971 tablet_replica.cc:442] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3: This TxnStatusTablet replica's current role is: LEADER
I20260620 09:21:08.425917 5995 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 09:21:08.426046 5995 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 09:21:08.426128 5447 catalog_manager.cc:5495] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 176c1ee9098d4fef911112f3b54995b3 (127.4.59.1). New cstate: current_term: 1 leader_uuid: "176c1ee9098d4fef911112f3b54995b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } health_report { overall_health: HEALTHY } } }
I20260620 09:21:08.433907 5952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:08.431570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57512 (local address 127.4.59.1:39233)
0620 09:21:08.431609 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:08.431611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:08.431634 (+ 23us) server_negotiation.cc:407] Connection header received
0620 09:21:08.431727 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:08.431728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:08.431739 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:08.431817 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:21:08.431945 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.432410 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.433175 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.433381 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.433468 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:08.433648 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:21:08.433689 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:21:08.433755 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:08.433823 (+ 68us) server_negotiation.cc:299] Negotiation successful
0620 09:21:08.433844 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 09:21:08.433941 5990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:08.431483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:39233 (local address 127.0.0.1:57512)
0620 09:21:08.431575 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:08.431588 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:08.431633 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:08.431827 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:08.431829 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:08.431830 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 09:21:08.431878 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:08.431880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.432491 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.432492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:08.433072 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:08.433074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.433469 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.433470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:08.433534 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:08.433556 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 09:21:08.433757 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 09:21:08.433766 (+ 9us) client_negotiation.cc:769] Sending connection context
0620 09:21:08.433818 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 09:21:08.433866 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260620 09:21:08.434968 5744 raft_consensus.cc:1261] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36 [term 1 FOLLOWER]: Refusing update from remote peer 176c1ee9098d4fef911112f3b54995b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:08.435030 5884 raft_consensus.cc:1261] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b [term 1 FOLLOWER]: Refusing update from remote peer 176c1ee9098d4fef911112f3b54995b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:08.435155 5962 consensus_queue.cc:1035] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:08.435236 5971 consensus_queue.cc:1035] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:08.435663 5960 tablet_replica.cc:439] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36: TxnStatusTablet state changed. Reason: New leader 176c1ee9098d4fef911112f3b54995b3. Latest consensus state: current_term: 1 leader_uuid: "176c1ee9098d4fef911112f3b54995b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } }
I20260620 09:21:08.435763 5960 tablet_replica.cc:442] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:21:08.435860 5961 tablet_replica.cc:439] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b: TxnStatusTablet state changed. Reason: New leader 176c1ee9098d4fef911112f3b54995b3. Latest consensus state: current_term: 1 leader_uuid: "176c1ee9098d4fef911112f3b54995b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } }
I20260620 09:21:08.435927 5961 tablet_replica.cc:442] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:21:08.436184 5962 tablet_replica.cc:439] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "176c1ee9098d4fef911112f3b54995b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } }
I20260620 09:21:08.436260 5962 tablet_replica.cc:442] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3: This TxnStatusTablet replica's current role is: LEADER
I20260620 09:21:08.436753 5960 tablet_replica.cc:439] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "176c1ee9098d4fef911112f3b54995b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } }
I20260620 09:21:08.436825 5960 tablet_replica.cc:442] T 2d07051c4b234dbbaad96f9cdde56d76 P ec80fac3256a42f0a14a3c8efbd92a36: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:21:08.436828 5959 tablet_replica.cc:439] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "176c1ee9098d4fef911112f3b54995b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } }
I20260620 09:21:08.436910 5959 tablet_replica.cc:442] T 2d07051c4b234dbbaad96f9cdde56d76 P 176c1ee9098d4fef911112f3b54995b3: This TxnStatusTablet replica's current role is: LEADER
I20260620 09:21:08.437454 5968 mvcc.cc:205] Tried to move back new op lower bound from 7298856011508047872 to 7298856011470708736. Current Snapshot: MvccSnapshot[applied={T|T < 7298856011508047872}]
I20260620 09:21:08.437726 5961 tablet_replica.cc:439] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "176c1ee9098d4fef911112f3b54995b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec80fac3256a42f0a14a3c8efbd92a36" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36283 } } peers { permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 } } peers { permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 } } }
I20260620 09:21:08.437805 5961 tablet_replica.cc:442] T 2d07051c4b234dbbaad96f9cdde56d76 P 9f5fba8480ef4692a1e8a91f9c9d904b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:21:08.440666 5949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:08.438406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45504 (local address 127.4.59.2:36283)
0620 09:21:08.438442 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:08.438444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:08.438451 (+ 7us) server_negotiation.cc:407] Connection header received
0620 09:21:08.438517 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:08.438517 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:08.438535 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:08.438599 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:21:08.438703 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.439155 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.439882 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.440092 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.440177 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:08.440334 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:21:08.440438 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:21:08.440520 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:08.440565 (+ 45us) server_negotiation.cc:299] Negotiation successful
0620 09:21:08.440585 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:21:08.445463 5952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:08.442879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:35925 (local address 127.4.59.1:39233)
0620 09:21:08.442913 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:08.442915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:08.443075 (+ 160us) server_negotiation.cc:407] Connection header received
0620 09:21:08.443102 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:08.443102 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:08.443113 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:08.443160 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:21:08.443301 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.443748 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.444815 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.444998 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.445054 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:08.445256 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:21:08.445283 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:21:08.445332 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:08.445399 (+ 67us) server_negotiation.cc:299] Negotiation successful
0620 09:21:08.445418 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 09:21:08.445528 6002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:08.442800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:39233 (local address 127.4.59.2:35925)
0620 09:21:08.442952 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:08.442962 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:08.443018 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:08.443182 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:08.443183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:08.443184 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 09:21:08.443241 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:08.443242 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.443818 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.443819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:08.444729 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:08.444731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.445073 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.445074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:08.445161 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:08.445177 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 09:21:08.445346 (+ 169us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 09:21:08.445348 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:21:08.445398 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 09:21:08.445439 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260620 09:21:08.447587 5604 raft_consensus.cc:1261] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3 [term 1 FOLLOWER]: Refusing update from remote peer ec80fac3256a42f0a14a3c8efbd92a36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:08.447592 5884 raft_consensus.cc:1261] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b [term 1 FOLLOWER]: Refusing update from remote peer ec80fac3256a42f0a14a3c8efbd92a36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:08.447989 5983 consensus_queue.cc:1035] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f5fba8480ef4692a1e8a91f9c9d904b" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:08.448194 5983 consensus_queue.cc:1035] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "176c1ee9098d4fef911112f3b54995b3" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:08.455724 6006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:08.453047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:36283 (local address 127.4.59.1:37571)
0620 09:21:08.453200 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:08.453210 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:08.453277 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:08.453411 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:08.453412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:08.453413 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 09:21:08.453446 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:08.453447 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.454248 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.454248 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:08.454969 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:08.454971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.455404 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.455405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:08.455442 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:08.455453 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 09:21:08.455620 (+ 167us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 09:21:08.455621 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 09:21:08.455648 (+ 27us) client_negotiation.cc:240] Negotiation successful
0620 09:21:08.455676 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
I20260620 09:21:08.455739 5949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:08.453106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:37571 (local address 127.4.59.2:36283)
0620 09:21:08.453183 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:08.453186 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:08.453253 (+ 67us) server_negotiation.cc:407] Connection header received
0620 09:21:08.453340 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:08.453342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:08.453353 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:08.453411 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:21:08.453484 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.454176 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.455058 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.455325 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.455359 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:08.455528 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:21:08.455561 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:21:08.455622 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:08.455655 (+ 33us) server_negotiation.cc:299] Negotiation successful
0620 09:21:08.455673 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 09:21:08.465608 5447 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:08.465725 5447 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:08.466332 5447 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81210b6086ea48219e467afceb2756da: Sending DeleteTablet for 3 replicas of tablet 734ebf7d1e914a798f3861351690e49c
I20260620 09:21:08.466599 5864 tablet_service.cc:1514] Processing DeleteTablet for tablet 734ebf7d1e914a798f3861351690e49c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:08 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:47046
I20260620 09:21:08.466728 5977 tablet_replica.cc:330] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b: stopping tablet replica
I20260620 09:21:08.466723 5584 tablet_service.cc:1514] Processing DeleteTablet for tablet 734ebf7d1e914a798f3861351690e49c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:08 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:57496
I20260620 09:21:08.466802 5977 raft_consensus.cc:2229] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:08.466859 5977 raft_consensus.cc:2258] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:08.466866 5724 tablet_service.cc:1514] Processing DeleteTablet for tablet 734ebf7d1e914a798f3861351690e49c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:08 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:45492
I20260620 09:21:08.466936 5976 tablet_replica.cc:330] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3: stopping tablet replica
I20260620 09:21:08.466948 5978 tablet_replica.cc:330] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36: stopping tablet replica
I20260620 09:21:08.466991 5976 raft_consensus.cc:2229] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:08.466995 5978 raft_consensus.cc:2229] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:21:08.467026 5976 raft_consensus.cc:2258] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:08.467068 5978 raft_consensus.cc:2258] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:08.467404 5976 ts_tablet_manager.cc:1905] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:08.467409 5978 ts_tablet_manager.cc:1905] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:08.469051 5977 ts_tablet_manager.cc:1905] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:08.469112 5978 ts_tablet_manager.cc:1918] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 09:21:08.469188 5978 log.cc:1196] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/734ebf7d1e914a798f3861351690e49c
I20260620 09:21:08.469429 5978 ts_tablet_manager.cc:1939] T 734ebf7d1e914a798f3861351690e49c P ec80fac3256a42f0a14a3c8efbd92a36: Deleting consensus metadata
I20260620 09:21:08.469735 5433 catalog_manager.cc:4841] TS ec80fac3256a42f0a14a3c8efbd92a36 (127.4.59.2:36283): tablet 734ebf7d1e914a798f3861351690e49c (table test-table [id=6b17f54141f0497b87d809423089763c]) successfully deleted
I20260620 09:21:08.470521 5977 ts_tablet_manager.cc:1918] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 09:21:08.470583 5977 log.cc:1196] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/734ebf7d1e914a798f3861351690e49c
I20260620 09:21:08.470790 5977 ts_tablet_manager.cc:1939] T 734ebf7d1e914a798f3861351690e49c P 9f5fba8480ef4692a1e8a91f9c9d904b: Deleting consensus metadata
I20260620 09:21:08.470974 5976 ts_tablet_manager.cc:1918] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 09:21:08.471026 5976 log.cc:1196] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781947262065055-4332-0/minicluster-data/ts-0/wal/wals/734ebf7d1e914a798f3861351690e49c
I20260620 09:21:08.471225 5976 ts_tablet_manager.cc:1939] T 734ebf7d1e914a798f3861351690e49c P 176c1ee9098d4fef911112f3b54995b3: Deleting consensus metadata
I20260620 09:21:08.471213 5434 catalog_manager.cc:4841] TS 9f5fba8480ef4692a1e8a91f9c9d904b (127.4.59.3:38667): tablet 734ebf7d1e914a798f3861351690e49c (table test-table [id=6b17f54141f0497b87d809423089763c]) successfully deleted
I20260620 09:21:08.471514 5433 catalog_manager.cc:4841] TS 176c1ee9098d4fef911112f3b54995b3 (127.4.59.1:39233): tablet 734ebf7d1e914a798f3861351690e49c (table test-table [id=6b17f54141f0497b87d809423089763c]) successfully deleted
Jun 20 09:21:08 dist-test-slave-414w krb5kdc[5402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947267, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.59.1@KRBTEST.COM
I20260620 09:21:08.477869 5952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:08.468750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57522 (local address 127.4.59.1:39233)
0620 09:21:08.468799 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:08.468801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:08.468914 (+ 113us) server_negotiation.cc:407] Connection header received
0620 09:21:08.469017 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:08.469018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:08.469027 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:08.469075 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:08.470089 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.470543 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.471864 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.472072 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.472154 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:08.474995 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:08.475003 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:08.475004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:08.475011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:08.476620 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:08.477002 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:08.477005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:08.477006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:08.477072 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:08.477305 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:08.477306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:08.477307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:08.477445 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:08.477583 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:08.477672 (+ 89us) server_negotiation.cc:299] Negotiation successful
0620 09:21:08.477746 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260620 09:21:08.478184 5644 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57522
I20260620 09:21:08.485879 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:08.479053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59284 (local address 127.4.59.62:33003)
0620 09:21:08.479094 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:08.479096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:08.479277 (+ 181us) server_negotiation.cc:407] Connection header received
0620 09:21:08.479354 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:08.479355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:08.479365 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:08.479423 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:08.480524 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.481081 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.481858 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:08.482068 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:08.482151 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:08.483291 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:08.483293 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:08.483293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:08.483300 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:08.484700 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:08.485064 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:08.485065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:08.485066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:08.485181 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:08.485379 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:08.485379 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:08.485380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:08.485492 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:08.485597 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:08.485698 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 09:21:08.485772 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 09:21:08.486080 5447 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59284
I20260620 09:21:08.486680 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 5519
I20260620 09:21:08.494278 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 5659
I20260620 09:21:08.500296 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 5799
I20260620 09:21:08.504482 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 5415
2026-06-20T09:21:08Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2411 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:21:08 dist-test-slave-414w krb5kdc[6016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:21:08 dist-test-slave-414w krb5kdc[6016](info): set up 2 sockets
Jun 20 09:21:08 dist-test-slave-414w krb5kdc[6016](info): commencing operation
krb5kdc: starting...
W20260620 09:21:09.545265 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:21:09 dist-test-slave-414w krb5kdc[6016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:21:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:21:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:21:09.663770 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:40191
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:35989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:40191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:09.743350 6029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:09.743563 6029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:09.743587 6029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:09.744666 6029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:09.744700 6029 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 09:21:09.744714 6029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:09.744726 6029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:09.744738 6029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:09.744750 6029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:21:09.746129 6029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:35989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:40191
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:40191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:09.746305 6029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:09.746479 6029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:09.751669 6035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:09.751735 6037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:09.752086 6029 server_base.cc:1034] running on GCE node
W20260620 09:21:09.752204 6034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:09.752444 6029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:09.752744 6029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:09.753928 6029 hybrid_clock.cc:648] HybridClock initialized: now 1781947269753913 us; error 47 us; skew 500 ppm
Jun 20 09:21:09 dist-test-slave-414w krb5kdc[6016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947269, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:09.756435 6029 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:21:09.756938 6029 webserver.cc:460] Webserver started at http://127.4.59.62:37303/ using document root <none> and password file <none>
I20260620 09:21:09.757233 6029 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:09.757305 6029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:09.757450 6029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:09.758486 6029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "81f5c4b666834bd9acd7cb6be12fd24e"
format_stamp: "Formatted at 2026-06-20 09:21:09 on dist-test-slave-414w"
server_key: "bf7c2a838f7ac0f47c3063e224b594fe"
server_key_iv: "febe34f4c293d82af9b67e3e4fd701c7"
server_key_version: "encryptionkey@0"
I20260620 09:21:09.758858 6029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "81f5c4b666834bd9acd7cb6be12fd24e"
format_stamp: "Formatted at 2026-06-20 09:21:09 on dist-test-slave-414w"
server_key: "bf7c2a838f7ac0f47c3063e224b594fe"
server_key_iv: "febe34f4c293d82af9b67e3e4fd701c7"
server_key_version: "encryptionkey@0"
I20260620 09:21:09.760190 6029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:21:09.760995 6044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:09.761269 6029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:21:09.761372 6029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "81f5c4b666834bd9acd7cb6be12fd24e"
format_stamp: "Formatted at 2026-06-20 09:21:09 on dist-test-slave-414w"
server_key: "bf7c2a838f7ac0f47c3063e224b594fe"
server_key_iv: "febe34f4c293d82af9b67e3e4fd701c7"
server_key_version: "encryptionkey@0"
I20260620 09:21:09.761453 6029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:09.778360 6029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:09.780678 6029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:09.780849 6029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:09.785573 6029 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:40191
I20260620 09:21:09.785622 6106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:40191 every 8 connection(s)
I20260620 09:21:09.785943 6029 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:21:09.786536 6107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:09.788395 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 6029
I20260620 09:21:09.788467 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:21:09.788653 4332 external_mini_cluster.cc:1428] Setting key 955600a9a550eade561a49c80e9fbed4
I20260620 09:21:09.788877 6107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e: Bootstrap starting.
I20260620 09:21:09.789551 6107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:09.789840 6107 log.cc:824] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:09.790503 6107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e: No bootstrap required, opened a new log
I20260620 09:21:09.792593 6107 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f5c4b666834bd9acd7cb6be12fd24e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40191 } }
I20260620 09:21:09.792790 6107 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:09.792832 6107 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81f5c4b666834bd9acd7cb6be12fd24e, State: Initialized, Role: FOLLOWER
I20260620 09:21:09.792996 6107 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f5c4b666834bd9acd7cb6be12fd24e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40191 } }
I20260620 09:21:09.793116 6107 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:21:09.793202 6107 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:21:09.793259 6107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:09.794027 6107 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f5c4b666834bd9acd7cb6be12fd24e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40191 } }
I20260620 09:21:09.794221 6107 leader_election.cc:302] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81f5c4b666834bd9acd7cb6be12fd24e; no voters:
I20260620 09:21:09.794394 6107 leader_election.cc:288] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:21:09.794564 6113 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:09.794656 6107 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:09.795059 6113 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [term 1 LEADER]: Becoming Leader. State: Replica: 81f5c4b666834bd9acd7cb6be12fd24e, State: Running, Role: LEADER
Jun 20 09:21:09 dist-test-slave-414w krb5kdc[6016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:09.795351 6113 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f5c4b666834bd9acd7cb6be12fd24e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40191 } }
I20260620 09:21:09.797322 6114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81f5c4b666834bd9acd7cb6be12fd24e. Latest consensus state: current_term: 1 leader_uuid: "81f5c4b666834bd9acd7cb6be12fd24e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f5c4b666834bd9acd7cb6be12fd24e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40191 } } }
I20260620 09:21:09.797341 6112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81f5c4b666834bd9acd7cb6be12fd24e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f5c4b666834bd9acd7cb6be12fd24e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40191 } } }
W20260620 09:21:09.797948 6126 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260620 09:21:09.797983 6112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [sys.catalog]: This master's current role is: LEADER
W20260620 09:21:09.798002 6126 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:21:09.798224 6114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:09.798616 6128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:21:09.799248 6128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:21:09.801216 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:09.789125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45542 (local address 127.4.59.62:40191)
0620 09:21:09.789357 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:09.789372 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:09.789397 (+ 25us) server_negotiation.cc:407] Connection header received
0620 09:21:09.789903 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:09.789909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:09.789983 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:09.790135 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:09.790793 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:09.791676 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:09.792867 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:09.793522 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:09.793656 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:09.797234 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:09.797239 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:09.797240 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:09.797253 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:09.799503 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:09.799918 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:09.799921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:09.799922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:09.800030 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:09.800255 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:09.800256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:09.800256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:09.800556 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:09.800718 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:09.800852 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 09:21:09.800990 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
I20260620 09:21:09.805578 6128 catalog_manager.cc:1347] Generated new cluster ID: aeeea260fe0f4821b3f52216f5b3e3a3
I20260620 09:21:09.805640 6128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:21:09.827451 6128 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:21:09.828198 6128 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:21:09.834532 6128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e: Generated new TSK 0
I20260620 09:21:09.834734 6128 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:21:09.869351 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:40191
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:35989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:09.973728 6133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:09.973922 6133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:09.973946 6133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:09.975255 6133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:09.975307 6133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:09.975329 6133 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 09:21:09.975342 6133 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 09:21:09.975399 6133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:21:09.977210 6133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:35989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.59.62:40191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:09.977447 6133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:09.977676 6133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:09.982620 6138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:09.982610 6139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:09.982739 6133 server_base.cc:1034] running on GCE node
W20260620 09:21:09.982599 6141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:09.982968 6133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:09.983294 6133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:09.984481 6133 hybrid_clock.cc:648] HybridClock initialized: now 1781947269984463 us; error 51 us; skew 500 ppm
Jun 20 09:21:09 dist-test-slave-414w krb5kdc[6016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947269, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:09.987012 6133 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:21:09.987454 6133 webserver.cc:460] Webserver started at http://127.4.59.1:44809/ using document root <none> and password file <none>
I20260620 09:21:09.987637 6133 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:09.987679 6133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:09.987764 6133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:09.988703 6133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "933db276f19a490199a7496e848c03af"
format_stamp: "Formatted at 2026-06-20 09:21:09 on dist-test-slave-414w"
server_key: "55d76f948717c552ceb2073866c632f0"
server_key_iv: "058f6ecc5d40cdb197a2ad739e124db5"
server_key_version: "encryptionkey@0"
I20260620 09:21:09.988987 6133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "933db276f19a490199a7496e848c03af"
format_stamp: "Formatted at 2026-06-20 09:21:09 on dist-test-slave-414w"
server_key: "55d76f948717c552ceb2073866c632f0"
server_key_iv: "058f6ecc5d40cdb197a2ad739e124db5"
server_key_version: "encryptionkey@0"
I20260620 09:21:09.990275 6133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:21:09.990926 6148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:09.991110 6133 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:09.991178 6133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "933db276f19a490199a7496e848c03af"
format_stamp: "Formatted at 2026-06-20 09:21:09 on dist-test-slave-414w"
server_key: "55d76f948717c552ceb2073866c632f0"
server_key_iv: "058f6ecc5d40cdb197a2ad739e124db5"
server_key_version: "encryptionkey@0"
I20260620 09:21:09.991227 6133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:10.004081 6133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:10.006248 6133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:10.006351 6133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:10.006865 6133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:10.006912 6133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:10.006953 6133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:10.006994 6133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 09:21:10 dist-test-slave-414w krb5kdc[6016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947269, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:10.014176 6133 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:39277
I20260620 09:21:10.014494 6133 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:21:10.014514 6263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:39277 every 8 connection(s)
I20260620 09:21:10.015409 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 6133
I20260620 09:21:10.015488 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:21:10.015679 4332 external_mini_cluster.cc:1428] Setting key 7ffd45bead3def78e4982d124cec18da
I20260620 09:21:10.017179 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.007238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:58555 (local address 127.4.59.62:40191)
0620 09:21:10.007302 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.007304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.007562 (+ 258us) server_negotiation.cc:407] Connection header received
0620 09:21:10.008247 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.008248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.008257 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.008300 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:10.009203 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.009698 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.010711 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.010923 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.010995 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.013895 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:10.013896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:10.013896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:10.013902 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:10.015553 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:10.016162 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:10.016163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:10.016164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:10.016223 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:10.016468 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:10.016469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:10.016469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:10.016622 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:10.016693 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.016913 (+ 220us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.017013 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 09:21:10.017215 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.007225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40191 (local address 127.4.59.1:58555)
0620 09:21:10.007408 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.007457 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.008134 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.008382 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.008386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.008788 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:10.009078 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.009085 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.009799 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.009800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.010619 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.010621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.011041 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.011042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.011148 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.011636 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:10.011645 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:10.013808 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:10.015619 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:10.015625 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:10.015627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:10.015919 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:10.016294 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:10.016295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:10.016295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:10.016404 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:10.016687 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:10.016691 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:10.016793 (+ 102us) client_negotiation.cc:769] Sending connection context
0620 09:21:10.016872 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 09:21:10.016983 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":110,"threads_started":1}
I20260620 09:21:10.018468 6264 heartbeater.cc:347] Connected to a master server at 127.4.59.62:40191
I20260620 09:21:10.018551 6264 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:10.018733 6264 heartbeater.cc:511] Master 127.4.59.62:40191 requested a full tablet report, sending...
I20260620 09:21:10.019199 6060 ts_manager.cc:194] Registered new tserver with Master: 933db276f19a490199a7496e848c03af (127.4.59.1:39277)
I20260620 09:21:10.020217 6060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:58555
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
I20260620 09:21:10.024880 6271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.018594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40191 (local address 127.4.59.1:49045)
0620 09:21:10.018729 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.018761 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.018836 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.019020 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.019021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.019172 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:10.019219 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.019220 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.020186 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.020187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.020977 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.020979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.021518 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.021519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.021570 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.021976 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:10.021980 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:10.022252 (+ 272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:10.023937 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:10.023938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:10.023938 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:10.024105 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:10.024297 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:10.024298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:10.024298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:10.024345 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:10.024654 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:10.024655 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:10.024688 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 09:21:10.024743 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 09:21:10.024812 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260620 09:21:10.024884 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.018595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:49045 (local address 127.4.59.62:40191)
0620 09:21:10.018633 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.018635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.018813 (+ 178us) server_negotiation.cc:407] Connection header received
0620 09:21:10.018891 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.018892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.018903 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.018974 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:10.019361 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.020097 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.021072 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.021438 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.021523 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.022338 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:10.022339 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:10.022339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:10.022346 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:10.023851 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:10.024177 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:10.024178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:10.024179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:10.024233 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:10.024455 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:10.024456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:10.024456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:10.024583 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:10.024658 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.024771 (+ 113us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.024828 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":208,"server-negotiator.queue_time_us":21}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:21:10.040120 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:40191
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:35989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:10.122617 6273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:10.122805 6273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:10.122828 6273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:10.123945 6273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:10.123984 6273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:10.124004 6273 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 09:21:10.124017 6273 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 09:21:10.124033 6273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:21:10.125952 6273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:35989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.59.62:40191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:10.126142 6273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:10.126310 6273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:10.130926 6281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:10.130934 6279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:10.131069 6273 server_base.cc:1034] running on GCE node
W20260620 09:21:10.130920 6278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:10.131330 6273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:10.131580 6273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:10.132766 6273 hybrid_clock.cc:648] HybridClock initialized: now 1781947270132748 us; error 27 us; skew 500 ppm
Jun 20 09:21:10 dist-test-slave-414w krb5kdc[6016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947270, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:10.135473 6273 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:21:10.135932 6273 webserver.cc:460] Webserver started at http://127.4.59.2:43245/ using document root <none> and password file <none>
I20260620 09:21:10.136173 6273 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:10.136241 6273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:10.136392 6273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:10.137463 6273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "8cc7b33592094eacb715d0443a80fb9c"
format_stamp: "Formatted at 2026-06-20 09:21:10 on dist-test-slave-414w"
server_key: "4db035ad95dabd5e162582301c2bd7d6"
server_key_iv: "03b69f93613dc34409b8d042b6384205"
server_key_version: "encryptionkey@0"
I20260620 09:21:10.137801 6273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "8cc7b33592094eacb715d0443a80fb9c"
format_stamp: "Formatted at 2026-06-20 09:21:10 on dist-test-slave-414w"
server_key: "4db035ad95dabd5e162582301c2bd7d6"
server_key_iv: "03b69f93613dc34409b8d042b6384205"
server_key_version: "encryptionkey@0"
I20260620 09:21:10.139075 6273 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:10.139750 6288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:10.139910 6273 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:10.140014 6273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "8cc7b33592094eacb715d0443a80fb9c"
format_stamp: "Formatted at 2026-06-20 09:21:10 on dist-test-slave-414w"
server_key: "4db035ad95dabd5e162582301c2bd7d6"
server_key_iv: "03b69f93613dc34409b8d042b6384205"
server_key_version: "encryptionkey@0"
I20260620 09:21:10.140084 6273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:10.152823 6273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:10.155048 6273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:10.155205 6273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:10.155730 6273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:10.155786 6273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:10.155818 6273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:10.155848 6273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 09:21:10 dist-test-slave-414w krb5kdc[6016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947270, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:10.163309 6273 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:36649
I20260620 09:21:10.163419 6403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:36649 every 8 connection(s)
I20260620 09:21:10.163725 6273 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:21:10.164532 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 6273
I20260620 09:21:10.164623 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:21:10.164809 4332 external_mini_cluster.cc:1428] Setting key 679a1f87bff097743c0fa81a3601fdfc
I20260620 09:21:10.166857 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.156117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:52807 (local address 127.4.59.62:40191)
0620 09:21:10.156183 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.156186 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.156423 (+ 237us) server_negotiation.cc:407] Connection header received
0620 09:21:10.156877 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.156878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.156890 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.156972 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:10.157829 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.158920 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.160543 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.160899 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.160990 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.163763 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:10.163764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:10.163764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:10.163772 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:10.165447 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:10.165908 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:10.165910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:10.165910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:10.165965 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:10.166251 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:10.166251 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:10.166252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:10.166363 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:10.166417 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.166729 (+ 312us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.166794 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 09:21:10.167135 6297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.156098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40191 (local address 127.4.59.2:52807)
0620 09:21:10.156280 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.156319 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.156778 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.157084 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.157088 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.157419 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:10.157661 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.157666 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.159062 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.159063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.160442 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.160445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.161018 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.161019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.161188 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.162161 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:10.162169 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:10.163652 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:10.165525 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:10.165531 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:10.165533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:10.165830 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:10.166036 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:10.166038 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:10.166038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:10.166180 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:10.166429 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:10.166434 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:10.166591 (+ 157us) client_negotiation.cc:769] Sending connection context
0620 09:21:10.166702 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 09:21:10.166874 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":116,"threads_started":1}
I20260620 09:21:10.168561 6404 heartbeater.cc:347] Connected to a master server at 127.4.59.62:40191
I20260620 09:21:10.168675 6404 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:10.168968 6404 heartbeater.cc:511] Master 127.4.59.62:40191 requested a full tablet report, sending...
I20260620 09:21:10.169463 6060 ts_manager.cc:194] Registered new tserver with Master: 8cc7b33592094eacb715d0443a80fb9c (127.4.59.2:36649)
I20260620 09:21:10.169894 6060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:52807
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
I20260620 09:21:10.174906 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.168773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:32989 (local address 127.4.59.62:40191)
0620 09:21:10.168810 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.168812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.168955 (+ 143us) server_negotiation.cc:407] Connection header received
0620 09:21:10.169013 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.169014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.169025 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.169071 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:10.169471 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.170285 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.171059 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.171246 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.171332 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.172213 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:10.172215 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:10.172215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:10.172222 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:10.173613 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:10.173969 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:10.173970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:10.173971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:10.174027 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:10.174346 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:10.174348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:10.174348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:10.174505 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:10.174588 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.174747 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.174819 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:21:10.174919 6411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.168735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40191 (local address 127.4.59.2:32989)
0620 09:21:10.168881 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.168891 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.168954 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.169085 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.169086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.169323 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:10.169389 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.169391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.170379 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.170380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.170933 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.170935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.171346 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.171347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.171413 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.171853 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:10.171857 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:10.172127 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:10.173714 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:10.173715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:10.173715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:10.173885 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:10.174197 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:10.174198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:10.174199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:10.174276 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:10.174579 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:10.174579 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:10.174625 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 09:21:10.174711 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 09:21:10.174814 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":98,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:21:10.191452 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:40191
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:35989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:10.272258 6413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:10.272441 6413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:10.272464 6413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:10.273633 6413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:10.273670 6413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:10.273690 6413 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 09:21:10.273703 6413 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 09:21:10.273743 6413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:21:10.275116 6413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:35989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.59.62:40191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:10.275350 6413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:10.275560 6413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:10.280294 6419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:10.280311 6418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:10.280465 6413 server_base.cc:1034] running on GCE node
W20260620 09:21:10.280289 6421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:10.280716 6413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:10.280980 6413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:10.282136 6413 hybrid_clock.cc:648] HybridClock initialized: now 1781947270282123 us; error 44 us; skew 500 ppm
Jun 20 09:21:10 dist-test-slave-414w krb5kdc[6016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947270, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:10.284495 6413 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:21:10.284921 6413 webserver.cc:460] Webserver started at http://127.4.59.3:41621/ using document root <none> and password file <none>
I20260620 09:21:10.285188 6413 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:10.285281 6413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:10.285414 6413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:10.286456 6413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "b1de9b9873804459bb33266f21ec6697"
format_stamp: "Formatted at 2026-06-20 09:21:10 on dist-test-slave-414w"
server_key: "3c8fb05002eb66340a903ab39f5aa818"
server_key_iv: "393e5d886dae8320a3f6dec579848b04"
server_key_version: "encryptionkey@0"
I20260620 09:21:10.286815 6413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "b1de9b9873804459bb33266f21ec6697"
format_stamp: "Formatted at 2026-06-20 09:21:10 on dist-test-slave-414w"
server_key: "3c8fb05002eb66340a903ab39f5aa818"
server_key_iv: "393e5d886dae8320a3f6dec579848b04"
server_key_version: "encryptionkey@0"
I20260620 09:21:10.288076 6413 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:21:10.288789 6428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:10.288977 6413 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:10.289076 6413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "b1de9b9873804459bb33266f21ec6697"
format_stamp: "Formatted at 2026-06-20 09:21:10 on dist-test-slave-414w"
server_key: "3c8fb05002eb66340a903ab39f5aa818"
server_key_iv: "393e5d886dae8320a3f6dec579848b04"
server_key_version: "encryptionkey@0"
I20260620 09:21:10.289179 6413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:10.305576 6413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:10.307782 6413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:10.307940 6413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:10.308426 6413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:10.308485 6413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:10.308537 6413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:10.308588 6413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 09:21:10 dist-test-slave-414w krb5kdc[6016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947270, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:10.315920 6413 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:32933
I20260620 09:21:10.316011 6543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:32933 every 8 connection(s)
I20260620 09:21:10.316344 6413 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:21:10.319626 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.309025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:54491 (local address 127.4.59.62:40191)
0620 09:21:10.309102 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.309104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.309357 (+ 253us) server_negotiation.cc:407] Connection header received
0620 09:21:10.309889 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.309890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.309902 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.309971 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:10.310826 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.311640 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.313507 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.313723 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.313809 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.316563 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:10.316565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:10.316565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:10.316572 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:10.318222 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:10.318629 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:10.318631 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:10.318632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:10.318689 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:10.319005 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:10.319006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:10.319007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:10.319119 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:10.319196 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.319463 (+ 267us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.319536 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 09:21:10.319799 6437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.309056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40191 (local address 127.4.59.3:54491)
0620 09:21:10.309243 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.309275 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.309794 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.310039 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.310043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.310385 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:10.310720 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.310725 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.311799 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.311801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.313390 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.313395 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.313824 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.313825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.313960 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.314543 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:10.314560 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:10.316450 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:10.318307 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:10.318314 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:10.318316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:10.318537 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:10.318757 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:10.318758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:10.318758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:10.318930 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:10.319188 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:10.319194 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:10.319327 (+ 133us) client_negotiation.cc:769] Sending connection context
0620 09:21:10.319434 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 09:21:10.319582 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":109,"threads_started":1}
I20260620 09:21:10.320286 6544 heartbeater.cc:347] Connected to a master server at 127.4.59.62:40191
I20260620 09:21:10.320369 6544 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:10.320535 6544 heartbeater.cc:511] Master 127.4.59.62:40191 requested a full tablet report, sending...
I20260620 09:21:10.320886 6061 ts_manager.cc:194] Registered new tserver with Master: b1de9b9873804459bb33266f21ec6697 (127.4.59.3:32933)
I20260620 09:21:10.321341 6061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:54491
I20260620 09:21:10.326184 6550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.321357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40191 (local address 127.4.59.3:60179)
0620 09:21:10.321519 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.321530 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.321592 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.321702 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.321703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.321888 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:10.321940 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.321942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.322563 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.322564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.323055 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.323057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.323388 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.323389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.323443 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.323869 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:10.323874 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:10.324147 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:10.325460 (+ 1313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:10.325460 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:10.325461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:10.325616 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:10.325762 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:10.325762 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:10.325762 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:10.325803 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:10.325957 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:10.325957 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:10.325986 (+ 29us) client_negotiation.cc:769] Sending connection context
0620 09:21:10.326051 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 09:21:10.326115 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":108,"threads_started":1}
I20260620 09:21:10.326259 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.321414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:60179 (local address 127.4.59.62:40191)
0620 09:21:10.321518 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.321519 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.321568 (+ 49us) server_negotiation.cc:407] Connection header received
0620 09:21:10.321650 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.321651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.321659 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.321697 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:10.322012 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.322438 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.323140 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.323310 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.323364 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.324212 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:10.324213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:10.324213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:10.324218 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:10.325396 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:10.325676 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:10.325677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:10.325677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:10.325721 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:10.325835 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:10.325836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:10.325836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:10.325913 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:10.325954 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.326093 (+ 139us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.326169 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260620 09:21:10.326256 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 6413
I20260620 09:21:10.326330 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:21:10.326488 4332 external_mini_cluster.cc:1428] Setting key 16a59a7a28c14c1e20ba1099b5708232
I20260620 09:21:10.327204 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 09:21:10 dist-test-slave-414w krb5kdc[6016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 09:21:10 dist-test-slave-414w krb5kdc[6016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:10.347980 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.341923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45548 (local address 127.4.59.62:40191)
0620 09:21:10.341978 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.341980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.341988 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:21:10.342030 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.342031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.342040 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.342110 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:10.342666 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.343127 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.343836 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.344004 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.344058 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.346015 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:10.346016 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:10.346016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:10.346022 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:10.347129 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:10.347415 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:10.347416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:10.347416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:10.347459 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:10.347611 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:10.347611 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:10.347611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:10.347698 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:10.347770 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.347850 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.347899 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 09:21:10.349586 6060 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:21:10.351049 6060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:21:10.357072 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.353376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:39277 (local address 127.0.0.1:35802)
0620 09:21:10.353701 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.353711 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.353765 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.353977 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.353978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.353980 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:10.354029 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.354030 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.355219 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.355220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.356092 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.356095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.356783 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.356784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.356859 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.356872 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 09:21:10.356941 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 09:21:10.356995 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":63,"threads_started":1}
I20260620 09:21:10.357095 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.353476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35802 (local address 127.4.59.1:39277)
0620 09:21:10.353651 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.353655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.353757 (+ 102us) server_negotiation.cc:407] Connection header received
0620 09:21:10.353829 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.353831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.353844 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.353912 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:10.354119 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.355140 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.356175 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.356683 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.356780 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.356838 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.356963 (+ 125us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.356996 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":71,"threads_started":1}
I20260620 09:21:10.357354 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.353108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:32933 (local address 127.0.0.1:42662)
0620 09:21:10.353270 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.353312 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.353376 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.353692 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.353694 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.353697 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:10.353756 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.353758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.355125 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.355126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.356168 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.356170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.357056 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.357057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.357112 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.357114 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:21:10.357223 (+ 109us) client_negotiation.cc:240] Negotiation successful
0620 09:21:10.357269 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":254,"threads_started":1}
I20260620 09:21:10.357364 6561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.353108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:36649 (local address 127.0.0.1:47972)
0620 09:21:10.353476 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.353485 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.353537 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.353785 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.353786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.353788 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:10.353888 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.353889 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.354760 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.354761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.355958 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.355961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.356908 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.356910 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.356980 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.356983 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:21:10.357029 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 09:21:10.357072 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":110,"threads_started":1}
I20260620 09:21:10.357599 6563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.353194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42662 (local address 127.4.59.3:32933)
0620 09:21:10.353517 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.353522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.353534 (+ 12us) server_negotiation.cc:407] Connection header received
0620 09:21:10.353574 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.353576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.353591 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.353681 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:10.353930 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.355040 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.356231 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.356922 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.356980 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.357026 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.357446 (+ 420us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.357497 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":74,"threads_started":1}
I20260620 09:21:10.357801 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.353221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47972 (local address 127.4.59.2:36649)
0620 09:21:10.353440 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.353444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.353573 (+ 129us) server_negotiation.cc:407] Connection header received
0620 09:21:10.353612 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.353614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.353628 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.353725 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:10.353964 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.354673 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.356031 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.356753 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.356821 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.356872 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.357618 (+ 746us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.357672 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":83,"threads_started":1}
I20260620 09:21:10.357775 6198 tablet_service.cc:1467] Processing CreateTablet for tablet 4ef304bd4b5a4a5a8f45a279efcc70af (DEFAULT_TABLE table=test-table [id=c4a752616ff34896b61c185d22bc6427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:10.358172 6198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ef304bd4b5a4a5a8f45a279efcc70af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:10.360770 6567 tablet_bootstrap.cc:492] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af: Bootstrap starting.
I20260620 09:21:10.361459 6567 tablet_bootstrap.cc:654] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:10.361403 6338 tablet_service.cc:1467] Processing CreateTablet for tablet 4ef304bd4b5a4a5a8f45a279efcc70af (DEFAULT_TABLE table=test-table [id=c4a752616ff34896b61c185d22bc6427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:10.361712 6338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ef304bd4b5a4a5a8f45a279efcc70af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:10.361687 6477 tablet_service.cc:1467] Processing CreateTablet for tablet 4ef304bd4b5a4a5a8f45a279efcc70af (DEFAULT_TABLE table=test-table [id=c4a752616ff34896b61c185d22bc6427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:10.361790 6567 log.cc:824] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:10.361974 6477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ef304bd4b5a4a5a8f45a279efcc70af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:10.362500 6567 tablet_bootstrap.cc:492] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af: No bootstrap required, opened a new log
I20260620 09:21:10.362581 6567 ts_tablet_manager.cc:1397] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:21:10.363875 6567 raft_consensus.cc:348] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } }
I20260620 09:21:10.364025 6567 raft_consensus.cc:374] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:10.364068 6567 raft_consensus.cc:729] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933db276f19a490199a7496e848c03af, State: Initialized, Role: FOLLOWER
I20260620 09:21:10.364223 6567 consensus_queue.cc:260] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } }
I20260620 09:21:10.364506 6264 heartbeater.cc:503] Master 127.4.59.62:40191 was elected leader, sending a full tablet report...
I20260620 09:21:10.364498 6567 ts_tablet_manager.cc:1428] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:21:10.365043 6571 tablet_bootstrap.cc:492] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c: Bootstrap starting.
I20260620 09:21:10.365309 6570 tablet_bootstrap.cc:492] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697: Bootstrap starting.
I20260620 09:21:10.365777 6571 tablet_bootstrap.cc:654] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:10.366076 6571 log.cc:824] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:10.366192 6570 tablet_bootstrap.cc:654] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:10.366528 6570 log.cc:824] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:10.366853 6571 tablet_bootstrap.cc:492] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c: No bootstrap required, opened a new log
I20260620 09:21:10.366925 6571 ts_tablet_manager.cc:1397] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:21:10.367240 6570 tablet_bootstrap.cc:492] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697: No bootstrap required, opened a new log
I20260620 09:21:10.367316 6570 ts_tablet_manager.cc:1397] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 09:21:10.368315 6571 raft_consensus.cc:348] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } }
I20260620 09:21:10.368427 6571 raft_consensus.cc:374] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:10.368448 6571 raft_consensus.cc:729] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc7b33592094eacb715d0443a80fb9c, State: Initialized, Role: FOLLOWER
I20260620 09:21:10.368552 6571 consensus_queue.cc:260] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } }
I20260620 09:21:10.368839 6571 ts_tablet_manager.cc:1428] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:21:10.368876 6404 heartbeater.cc:503] Master 127.4.59.62:40191 was elected leader, sending a full tablet report...
I20260620 09:21:10.368898 6570 raft_consensus.cc:348] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } }
I20260620 09:21:10.369040 6570 raft_consensus.cc:374] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:10.369072 6570 raft_consensus.cc:729] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1de9b9873804459bb33266f21ec6697, State: Initialized, Role: FOLLOWER
I20260620 09:21:10.369191 6570 consensus_queue.cc:260] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } }
I20260620 09:21:10.369434 6570 ts_tablet_manager.cc:1428] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:21:10.369547 6544 heartbeater.cc:503] Master 127.4.59.62:40191 was elected leader, sending a full tablet report...
W20260620 09:21:10.414536 6405 tablet.cc:2369] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 09:21:10.515446 6265 tablet.cc:2369] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 09:21:10.566974 6545 tablet.cc:2369] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:21:10.676329 6575 raft_consensus.cc:482] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:21:10.676455 6575 raft_consensus.cc:504] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } }
I20260620 09:21:10.676863 6575 leader_election.cc:288] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cc7b33592094eacb715d0443a80fb9c (127.4.59.2:36649), 933db276f19a490199a7496e848c03af (127.4.59.1:39277)
I20260620 09:21:10.679792 6437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.677085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:36649 (local address 127.4.59.3:59539)
0620 09:21:10.677168 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.677181 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.677243 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.677480 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.677481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.677483 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:10.677555 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.677557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.678267 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.678268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.679018 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.679019 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.679542 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.679544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.679614 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.679617 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:21:10.679665 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 09:21:10.679693 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":4}
I20260620 09:21:10.679803 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.677118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:59539 (local address 127.4.59.2:36649)
0620 09:21:10.677189 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.677191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.677248 (+ 57us) server_negotiation.cc:407] Connection header received
0620 09:21:10.677303 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.677304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.677315 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.677414 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:10.677631 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.678184 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.679104 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.679464 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.679515 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.679525 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.679685 (+ 160us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.679731 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 09:21:10.679850 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.677355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:52217 (local address 127.4.59.1:39277)
0620 09:21:10.677394 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.677396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.677489 (+ 93us) server_negotiation.cc:407] Connection header received
0620 09:21:10.677552 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.677553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.677564 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.677617 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:10.677756 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.678251 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.679146 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.679481 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.679567 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.679576 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.679738 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.679778 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 09:21:10.680078 6576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.677241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:39277 (local address 127.4.59.3:52217)
0620 09:21:10.677412 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.677424 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.677488 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.677627 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.677628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.677629 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:10.677691 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.677693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.678333 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.678333 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.679030 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.679032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.679587 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.679587 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.679644 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.679665 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 09:21:10.679722 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 09:21:10.679752 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":104,"threads_started":1}
I20260620 09:21:10.680168 6358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ef304bd4b5a4a5a8f45a279efcc70af" candidate_uuid: "b1de9b9873804459bb33266f21ec6697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cc7b33592094eacb715d0443a80fb9c" is_pre_election: true
I20260620 09:21:10.680307 6358 raft_consensus.cc:2454] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1de9b9873804459bb33266f21ec6697 in term 0.
I20260620 09:21:10.680573 6431 leader_election.cc:302] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cc7b33592094eacb715d0443a80fb9c, b1de9b9873804459bb33266f21ec6697; no voters:
I20260620 09:21:10.680656 6218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ef304bd4b5a4a5a8f45a279efcc70af" candidate_uuid: "b1de9b9873804459bb33266f21ec6697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933db276f19a490199a7496e848c03af" is_pre_election: true
I20260620 09:21:10.680697 6575 raft_consensus.cc:2781] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:21:10.680750 6575 raft_consensus.cc:482] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:21:10.680796 6575 raft_consensus.cc:3037] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:10.680806 6218 raft_consensus.cc:2454] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1de9b9873804459bb33266f21ec6697 in term 0.
I20260620 09:21:10.681521 6575 raft_consensus.cc:504] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } }
I20260620 09:21:10.681643 6575 leader_election.cc:288] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [CANDIDATE]: Term 1 election: Requested vote from peers 8cc7b33592094eacb715d0443a80fb9c (127.4.59.2:36649), 933db276f19a490199a7496e848c03af (127.4.59.1:39277)
I20260620 09:21:10.681798 6218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ef304bd4b5a4a5a8f45a279efcc70af" candidate_uuid: "b1de9b9873804459bb33266f21ec6697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933db276f19a490199a7496e848c03af"
I20260620 09:21:10.681815 6358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ef304bd4b5a4a5a8f45a279efcc70af" candidate_uuid: "b1de9b9873804459bb33266f21ec6697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cc7b33592094eacb715d0443a80fb9c"
I20260620 09:21:10.681895 6218 raft_consensus.cc:3037] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:10.681891 6358 raft_consensus.cc:3037] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:10.682507 6358 raft_consensus.cc:2454] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1de9b9873804459bb33266f21ec6697 in term 1.
I20260620 09:21:10.682658 6218 raft_consensus.cc:2454] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1de9b9873804459bb33266f21ec6697 in term 1.
I20260620 09:21:10.682675 6431 leader_election.cc:302] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cc7b33592094eacb715d0443a80fb9c, b1de9b9873804459bb33266f21ec6697; no voters:
I20260620 09:21:10.682842 6575 raft_consensus.cc:2781] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:10.682984 6575 raft_consensus.cc:686] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [term 1 LEADER]: Becoming Leader. State: Replica: b1de9b9873804459bb33266f21ec6697, State: Running, Role: LEADER
I20260620 09:21:10.683077 6575 consensus_queue.cc:237] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } }
I20260620 09:21:10.683808 6061 catalog_manager.cc:5495] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1de9b9873804459bb33266f21ec6697 (127.4.59.3). New cstate: current_term: 1 leader_uuid: "b1de9b9873804459bb33266f21ec6697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } health_report { overall_health: UNKNOWN } } }
Jun 20 09:21:10 dist-test-slave-414w krb5kdc[6016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947269, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:10.736264 6584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.729715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40191 (local address 127.0.0.1:45562)
0620 09:21:10.729858 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.729868 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.729929 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.730055 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.730056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.730338 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:10.730380 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.730382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.731007 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.731008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.731521 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.731522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.731858 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.731859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.731895 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.732371 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:10.732375 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:10.733734 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:10.735219 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:10.735225 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:10.735227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:10.735414 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:10.735614 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:10.735615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:10.735615 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:10.735669 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:10.735921 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:10.735927 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:10.735997 (+ 70us) client_negotiation.cc:769] Sending connection context
0620 09:21:10.736068 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 09:21:10.736194 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260620 09:21:10.736377 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.729790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45562 (local address 127.4.59.62:40191)
0620 09:21:10.729879 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.729880 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.729939 (+ 59us) server_negotiation.cc:407] Connection header received
0620 09:21:10.729972 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.729972 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.729980 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.730036 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:10.730464 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.730943 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.731609 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.731788 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.731850 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.733820 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:10.733821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:10.733821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:10.733827 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:10.735136 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:10.735489 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:10.735490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:10.735490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:10.735543 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:10.735743 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:10.735744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:10.735744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:10.735850 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:10.735938 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.736112 (+ 174us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.736251 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":60}
I20260620 09:21:10.738303 6061 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:45562:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 09:21:10.738539 6061 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:21:10.739933 6338 tablet_service.cc:1467] Processing CreateTablet for tablet d3e9ab5fffef419cb1f29bad8671c3a1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66eba57ee3654fecb61da217a1e0137c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 09:21:10.739934 6198 tablet_service.cc:1467] Processing CreateTablet for tablet d3e9ab5fffef419cb1f29bad8671c3a1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66eba57ee3654fecb61da217a1e0137c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 09:21:10.740114 6198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e9ab5fffef419cb1f29bad8671c3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:10.740105 6338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e9ab5fffef419cb1f29bad8671c3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:10.741461 6571 tablet_bootstrap.cc:492] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c: Bootstrap starting.
I20260620 09:21:10.741495 6567 tablet_bootstrap.cc:492] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af: Bootstrap starting.
I20260620 09:21:10.741508 6477 tablet_service.cc:1467] Processing CreateTablet for tablet d3e9ab5fffef419cb1f29bad8671c3a1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66eba57ee3654fecb61da217a1e0137c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 09:21:10.741657 6477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e9ab5fffef419cb1f29bad8671c3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:10.741871 6567 tablet_bootstrap.cc:654] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:10.741871 6571 tablet_bootstrap.cc:654] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:10.742554 6571 tablet_bootstrap.cc:492] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c: No bootstrap required, opened a new log
I20260620 09:21:10.742616 6571 ts_tablet_manager.cc:1397] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:10.742743 6567 tablet_bootstrap.cc:492] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af: No bootstrap required, opened a new log
I20260620 09:21:10.742791 6567 ts_tablet_manager.cc:1397] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:10.742780 6571 raft_consensus.cc:348] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:10.742870 6571 raft_consensus.cc:374] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:10.742898 6571 raft_consensus.cc:729] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc7b33592094eacb715d0443a80fb9c, State: Initialized, Role: FOLLOWER
I20260620 09:21:10.742938 6571 consensus_queue.cc:260] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:10.742944 6567 raft_consensus.cc:348] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:10.743001 6567 raft_consensus.cc:374] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:10.743026 6567 raft_consensus.cc:729] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933db276f19a490199a7496e848c03af, State: Initialized, Role: FOLLOWER
I20260620 09:21:10.743036 6571 ts_tablet_manager.cc:1428] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:10.743067 6567 consensus_queue.cc:260] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:10.743182 6567 ts_tablet_manager.cc:1428] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:10.743211 6570 tablet_bootstrap.cc:492] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697: Bootstrap starting.
I20260620 09:21:10.743222 6569 tablet_replica.cc:439] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } }
I20260620 09:21:10.743288 6569 tablet_replica.cc:442] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:21:10.743290 6574 tablet_replica.cc:439] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } }
I20260620 09:21:10.743358 6574 tablet_replica.cc:442] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:21:10.743641 6570 tablet_bootstrap.cc:654] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:10.744364 6570 tablet_bootstrap.cc:492] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697: No bootstrap required, opened a new log
I20260620 09:21:10.744413 6570 ts_tablet_manager.cc:1397] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260620 09:21:10.744561 6570 raft_consensus.cc:348] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:10.744619 6570 raft_consensus.cc:374] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:10.744664 6570 raft_consensus.cc:729] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1de9b9873804459bb33266f21ec6697, State: Initialized, Role: FOLLOWER
I20260620 09:21:10.744719 6570 consensus_queue.cc:260] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:10.744807 6570 ts_tablet_manager.cc:1428] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:10.744855 6575 tablet_replica.cc:439] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } }
I20260620 09:21:10.744922 6575 tablet_replica.cc:442] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:21:10.924180 6569 raft_consensus.cc:482] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:21:10.924306 6569 raft_consensus.cc:504] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:10.924669 6569 leader_election.cc:288] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cc7b33592094eacb715d0443a80fb9c (127.4.59.2:36649), b1de9b9873804459bb33266f21ec6697 (127.4.59.3:32933)
I20260620 09:21:10.927780 6588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.924881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:36649 (local address 127.4.59.1:48589)
0620 09:21:10.925015 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.925025 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.925119 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.925487 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.925488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.925490 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:10.925563 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.925564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.926178 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.926179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.927012 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.927014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.927549 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.927550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.927624 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.927628 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:21:10.927680 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 09:21:10.927711 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":1,"thread_start_us":58,"threads_started":1}
I20260620 09:21:10.927826 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.924940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:48589 (local address 127.4.59.2:36649)
0620 09:21:10.924989 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.924991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.925124 (+ 133us) server_negotiation.cc:407] Connection header received
0620 09:21:10.925386 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.925387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.925394 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.925470 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:10.925652 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.926108 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.927106 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.927456 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.927509 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.927517 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.927705 (+ 188us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.927744 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 09:21:10.928050 6358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e9ab5fffef419cb1f29bad8671c3a1" candidate_uuid: "933db276f19a490199a7496e848c03af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cc7b33592094eacb715d0443a80fb9c" is_pre_election: true
I20260620 09:21:10.928159 6358 raft_consensus.cc:2454] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 933db276f19a490199a7496e848c03af in term 0.
I20260620 09:21:10.928260 6589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.924893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:32933 (local address 127.4.59.1:49195)
0620 09:21:10.925011 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.925022 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.925075 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.925450 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.925451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.925454 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:10.925536 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.925538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.926287 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.926288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.927013 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.927015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.927579 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.927580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.928077 (+ 497us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.928081 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:21:10.928150 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 09:21:10.928187 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":8,"thread_start_us":57,"threads_started":1}
I20260620 09:21:10.928284 6590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.924987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:49195 (local address 127.4.59.3:32933)
0620 09:21:10.925080 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.925082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.925092 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:21:10.925334 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.925335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.925351 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.925433 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:10.925628 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.926204 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.927104 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.927474 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.927564 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.927574 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.928174 (+ 600us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.928213 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":46,"threads_started":1}
I20260620 09:21:10.928444 6151 leader_election.cc:302] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cc7b33592094eacb715d0443a80fb9c, 933db276f19a490199a7496e848c03af; no voters:
I20260620 09:21:10.928648 6498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e9ab5fffef419cb1f29bad8671c3a1" candidate_uuid: "933db276f19a490199a7496e848c03af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1de9b9873804459bb33266f21ec6697" is_pre_election: true
I20260620 09:21:10.928740 6498 raft_consensus.cc:2454] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 933db276f19a490199a7496e848c03af in term 0.
I20260620 09:21:10.928833 6569 raft_consensus.cc:2781] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:21:10.928882 6569 raft_consensus.cc:482] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:21:10.928903 6569 raft_consensus.cc:3037] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:10.929584 6569 raft_consensus.cc:504] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:10.929731 6569 leader_election.cc:288] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [CANDIDATE]: Term 1 election: Requested vote from peers 8cc7b33592094eacb715d0443a80fb9c (127.4.59.2:36649), b1de9b9873804459bb33266f21ec6697 (127.4.59.3:32933)
I20260620 09:21:10.929879 6498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e9ab5fffef419cb1f29bad8671c3a1" candidate_uuid: "933db276f19a490199a7496e848c03af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1de9b9873804459bb33266f21ec6697"
I20260620 09:21:10.929975 6498 raft_consensus.cc:3037] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:10.929946 6358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e9ab5fffef419cb1f29bad8671c3a1" candidate_uuid: "933db276f19a490199a7496e848c03af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cc7b33592094eacb715d0443a80fb9c"
I20260620 09:21:10.930007 6358 raft_consensus.cc:3037] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:10.930742 6358 raft_consensus.cc:2454] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 933db276f19a490199a7496e848c03af in term 1.
I20260620 09:21:10.930743 6498 raft_consensus.cc:2454] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 933db276f19a490199a7496e848c03af in term 1.
I20260620 09:21:10.930943 6150 leader_election.cc:302] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 933db276f19a490199a7496e848c03af, b1de9b9873804459bb33266f21ec6697; no voters:
I20260620 09:21:10.931049 6569 raft_consensus.cc:2781] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:10.931178 6569 raft_consensus.cc:686] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [term 1 LEADER]: Becoming Leader. State: Replica: 933db276f19a490199a7496e848c03af, State: Running, Role: LEADER
I20260620 09:21:10.931289 6569 consensus_queue.cc:237] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:10.931703 6591 tablet_replica.cc:439] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af: TxnStatusTablet state changed. Reason: New leader 933db276f19a490199a7496e848c03af. Latest consensus state: current_term: 1 leader_uuid: "933db276f19a490199a7496e848c03af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } }
I20260620 09:21:10.931790 6591 tablet_replica.cc:442] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af: This TxnStatusTablet replica's current role is: LEADER
I20260620 09:21:10.932022 6593 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 09:21:10.932130 6593 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 09:21:10.932075 6061 catalog_manager.cc:5495] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af reported cstate change: term changed from 0 to 1, leader changed from <none> to 933db276f19a490199a7496e848c03af (127.4.59.1). New cstate: current_term: 1 leader_uuid: "933db276f19a490199a7496e848c03af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } health_report { overall_health: UNKNOWN } } }
I20260620 09:21:10.972369 6584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.969419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:39277 (local address 127.0.0.1:35808)
0620 09:21:10.969551 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.969561 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.969610 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.969783 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.969784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.969785 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 09:21:10.969831 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.969832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.970732 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.970732 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.971321 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.971322 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.971787 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.971788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.971832 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.971845 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 09:21:10.972215 (+ 370us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 09:21:10.972223 (+ 8us) client_negotiation.cc:769] Sending connection context
0620 09:21:10.972267 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 09:21:10.972307 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260620 09:21:10.972401 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.969487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35808 (local address 127.4.59.1:39277)
0620 09:21:10.969545 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.969547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.969595 (+ 48us) server_negotiation.cc:407] Connection header received
0620 09:21:10.969679 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.969680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.969702 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.969773 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:21:10.969891 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.970654 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.971406 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.971711 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.971783 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.971931 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:21:10.972057 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:21:10.972146 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.972289 (+ 143us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.972311 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 09:21:10.974479 6358 raft_consensus.cc:1261] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c [term 1 FOLLOWER]: Refusing update from remote peer 933db276f19a490199a7496e848c03af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:10.974495 6498 raft_consensus.cc:1261] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697 [term 1 FOLLOWER]: Refusing update from remote peer 933db276f19a490199a7496e848c03af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:10.974802 6591 consensus_queue.cc:1035] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:10.974941 6569 consensus_queue.cc:1035] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:10.976037 6575 tablet_replica.cc:439] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697: TxnStatusTablet state changed. Reason: New leader 933db276f19a490199a7496e848c03af. Latest consensus state: current_term: 1 leader_uuid: "933db276f19a490199a7496e848c03af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } }
I20260620 09:21:10.976168 6575 tablet_replica.cc:442] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:21:10.976200 6574 tablet_replica.cc:439] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c: TxnStatusTablet state changed. Reason: New leader 933db276f19a490199a7496e848c03af. Latest consensus state: current_term: 1 leader_uuid: "933db276f19a490199a7496e848c03af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } }
I20260620 09:21:10.976294 6574 tablet_replica.cc:442] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:21:10.977926 6591 tablet_replica.cc:439] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "933db276f19a490199a7496e848c03af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } }
I20260620 09:21:10.978021 6591 tablet_replica.cc:442] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af: This TxnStatusTablet replica's current role is: LEADER
I20260620 09:21:10.978125 6591 tablet_replica.cc:439] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "933db276f19a490199a7496e848c03af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } }
I20260620 09:21:10.978171 6591 tablet_replica.cc:442] T d3e9ab5fffef419cb1f29bad8671c3a1 P 933db276f19a490199a7496e848c03af: This TxnStatusTablet replica's current role is: LEADER
I20260620 09:21:10.978119 6575 tablet_replica.cc:439] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "933db276f19a490199a7496e848c03af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } }
I20260620 09:21:10.978207 6575 tablet_replica.cc:442] T d3e9ab5fffef419cb1f29bad8671c3a1 P b1de9b9873804459bb33266f21ec6697: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:21:10.978227 6596 mvcc.cc:205] Tried to move back new op lower bound from 7298856021908541440 to 7298856021734887424. Current Snapshot: MvccSnapshot[applied={T|T < 7298856021908541440}]
I20260620 09:21:10.978453 6598 mvcc.cc:205] Tried to move back new op lower bound from 7298856021908541440 to 7298856021734887424. Current Snapshot: MvccSnapshot[applied={T|T < 7298856021908541440}]
I20260620 09:21:10.979033 6574 tablet_replica.cc:439] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "933db276f19a490199a7496e848c03af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } } }
I20260620 09:21:10.979115 6574 tablet_replica.cc:442] T d3e9ab5fffef419cb1f29bad8671c3a1 P 8cc7b33592094eacb715d0443a80fb9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:21:10.979631 6597 mvcc.cc:205] Tried to move back new op lower bound from 7298856021908541440 to 7298856021734887424. Current Snapshot: MvccSnapshot[applied={T|T < 7298856021908541440 or (T in {7298856021908541440})}]
I20260620 09:21:10.983296 6590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.980346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42678 (local address 127.4.59.3:32933)
0620 09:21:10.980389 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.980391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.980577 (+ 186us) server_negotiation.cc:407] Connection header received
0620 09:21:10.980607 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.980608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.980625 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.980677 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:21:10.980824 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.981379 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.982498 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.982680 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.982738 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.982912 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:21:10.982992 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:21:10.983048 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.983118 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.983136 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 09:21:10.988358 6608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.985727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:39277 (local address 127.4.59.3:43811)
0620 09:21:10.985877 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.985887 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.985945 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.986061 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.986061 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.986063 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 09:21:10.986107 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.986108 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.986931 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.986931 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.987503 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.987504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.987945 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.987946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:10.987993 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.988005 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 09:21:10.988225 (+ 220us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 09:21:10.988227 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:21:10.988264 (+ 37us) client_negotiation.cc:240] Negotiation successful
0620 09:21:10.988297 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260620 09:21:10.988355 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.985801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:43811 (local address 127.4.59.1:39277)
0620 09:21:10.985867 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.985869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.985937 (+ 68us) server_negotiation.cc:407] Connection header received
0620 09:21:10.985985 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.985986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.985997 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.986054 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:21:10.986147 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.986854 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.987583 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.987869 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.987941 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:10.988096 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:21:10.988129 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:21:10.988182 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:10.988267 (+ 85us) server_negotiation.cc:299] Negotiation successful
0620 09:21:10.988285 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 09:21:10.990900 6358 raft_consensus.cc:1261] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c [term 1 FOLLOWER]: Refusing update from remote peer b1de9b9873804459bb33266f21ec6697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:10.990922 6218 raft_consensus.cc:1261] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af [term 1 FOLLOWER]: Refusing update from remote peer b1de9b9873804459bb33266f21ec6697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:10.991156 6575 consensus_queue.cc:1035] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:10.991336 6577 consensus_queue.cc:1035] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:11.001252 6590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.998251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:53139 (local address 127.4.59.3:32933)
0620 09:21:10.998325 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:10.998327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:10.998525 (+ 198us) server_negotiation.cc:407] Connection header received
0620 09:21:10.998575 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:10.998575 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:10.998586 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:10.998650 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:21:10.998799 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.999558 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.000496 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.000755 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.000840 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:11.000972 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:21:11.001003 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:21:11.001057 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:11.001101 (+ 44us) server_negotiation.cc:299] Negotiation successful
0620 09:21:11.001132 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 09:21:11.001262 6614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:10.998150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:32933 (local address 127.4.59.1:53139)
0620 09:21:10.998410 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:10.998441 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:10.998522 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:10.998662 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:10.998663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:10.998665 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 09:21:10.998736 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:10.998737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:10.999634 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:10.999636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:11.000409 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:11.000412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.000846 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.000847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:11.000885 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:11.000897 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 09:21:11.001061 (+ 164us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 09:21:11.001063 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:21:11.001097 (+ 34us) client_negotiation.cc:240] Negotiation successful
0620 09:21:11.001129 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":165,"threads_started":1}
I20260620 09:21:11.010005 6061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:11.010119 6061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:11.010857 6061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e: Sending DeleteTablet for 3 replicas of tablet 4ef304bd4b5a4a5a8f45a279efcc70af
I20260620 09:21:11.011161 6477 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ef304bd4b5a4a5a8f45a279efcc70af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:11 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:42662
I20260620 09:21:11.011230 6198 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ef304bd4b5a4a5a8f45a279efcc70af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:11 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:35802
I20260620 09:21:11.011610 6338 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ef304bd4b5a4a5a8f45a279efcc70af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:11 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:47972
I20260620 09:21:11.011675 6061 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:21:11.011807 6061 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:21:11.011821 6618 tablet_replica.cc:330] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af: stopping tablet replica
I20260620 09:21:11.011916 6618 raft_consensus.cc:2229] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:11.011962 6618 raft_consensus.cc:2258] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:11.012329 6618 ts_tablet_manager.cc:1905] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:11.013238 6619 tablet_replica.cc:330] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c: stopping tablet replica
I20260620 09:21:11.013324 6198 tablet_service.cc:1467] Processing CreateTablet for tablet e7c672c2a0f646f48f992925c200017c (DEFAULT_TABLE table=test-table [id=1697c7312dcb47f5a22d961dc361385f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:11.013345 6619 raft_consensus.cc:2229] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:11.013396 6619 raft_consensus.cc:2258] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:11.013427 6198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7c672c2a0f646f48f992925c200017c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:11.013718 6619 ts_tablet_manager.cc:1905] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:11.014724 6338 tablet_service.cc:1467] Processing CreateTablet for tablet e7c672c2a0f646f48f992925c200017c (DEFAULT_TABLE table=test-table [id=1697c7312dcb47f5a22d961dc361385f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:11.014829 6338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7c672c2a0f646f48f992925c200017c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:11.016316 6619 ts_tablet_manager.cc:1918] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 09:21:11.016376 6619 log.cc:1196] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/4ef304bd4b5a4a5a8f45a279efcc70af
I20260620 09:21:11.016625 6619 ts_tablet_manager.cc:1939] T 4ef304bd4b5a4a5a8f45a279efcc70af P 8cc7b33592094eacb715d0443a80fb9c: Deleting consensus metadata
I20260620 09:21:11.016637 6567 tablet_bootstrap.cc:492] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af: Bootstrap starting.
I20260620 09:21:11.016788 6618 ts_tablet_manager.cc:1918] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 09:21:11.016844 6618 log.cc:1196] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/wal/wals/4ef304bd4b5a4a5a8f45a279efcc70af
I20260620 09:21:11.016912 6567 tablet_bootstrap.cc:654] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:11.017095 6618 ts_tablet_manager.cc:1939] T 4ef304bd4b5a4a5a8f45a279efcc70af P 933db276f19a490199a7496e848c03af: Deleting consensus metadata
I20260620 09:21:11.017272 6571 tablet_bootstrap.cc:492] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c: Bootstrap starting.
I20260620 09:21:11.017535 6617 tablet_replica.cc:330] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697: stopping tablet replica
I20260620 09:21:11.017654 6617 raft_consensus.cc:2229] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:21:11.017655 6571 tablet_bootstrap.cc:654] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:11.017745 6617 raft_consensus.cc:2258] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:11.018124 6617 ts_tablet_manager.cc:1905] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:11.018751 6477 tablet_service.cc:1467] Processing CreateTablet for tablet e7c672c2a0f646f48f992925c200017c (DEFAULT_TABLE table=test-table [id=1697c7312dcb47f5a22d961dc361385f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:11.018762 6047 catalog_manager.cc:4841] TS 933db276f19a490199a7496e848c03af (127.4.59.1:39277): tablet 4ef304bd4b5a4a5a8f45a279efcc70af (table test-table [id=c4a752616ff34896b61c185d22bc6427]) successfully deleted
I20260620 09:21:11.018854 6477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7c672c2a0f646f48f992925c200017c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:11.020151 6570 tablet_bootstrap.cc:492] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697: Bootstrap starting.
I20260620 09:21:11.020570 6570 tablet_bootstrap.cc:654] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:11.020663 6047 catalog_manager.cc:4841] TS 8cc7b33592094eacb715d0443a80fb9c (127.4.59.2:36649): tablet 4ef304bd4b5a4a5a8f45a279efcc70af (table test-table [id=c4a752616ff34896b61c185d22bc6427]) successfully deleted
I20260620 09:21:11.020310 6617 ts_tablet_manager.cc:1918] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 09:21:11.020805 6617 log.cc:1196] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/4ef304bd4b5a4a5a8f45a279efcc70af
I20260620 09:21:11.021060 6617 ts_tablet_manager.cc:1939] T 4ef304bd4b5a4a5a8f45a279efcc70af P b1de9b9873804459bb33266f21ec6697: Deleting consensus metadata
I20260620 09:21:11.021430 6046 catalog_manager.cc:4841] TS b1de9b9873804459bb33266f21ec6697 (127.4.59.3:32933): tablet 4ef304bd4b5a4a5a8f45a279efcc70af (table test-table [id=c4a752616ff34896b61c185d22bc6427]) successfully deleted
I20260620 09:21:11.025534 6571 tablet_bootstrap.cc:492] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c: No bootstrap required, opened a new log
I20260620 09:21:11.025594 6571 ts_tablet_manager.cc:1397] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260620 09:21:11.025753 6567 tablet_bootstrap.cc:492] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af: No bootstrap required, opened a new log
I20260620 09:21:11.025802 6567 ts_tablet_manager.cc:1397] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260620 09:21:11.025770 6571 raft_consensus.cc:348] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:11.025847 6571 raft_consensus.cc:374] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:11.025871 6571 raft_consensus.cc:729] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc7b33592094eacb715d0443a80fb9c, State: Initialized, Role: FOLLOWER
I20260620 09:21:11.025914 6571 consensus_queue.cc:260] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:11.025940 6567 raft_consensus.cc:348] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:11.026000 6571 ts_tablet_manager.cc:1428] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:11.026016 6567 raft_consensus.cc:374] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:11.026037 6567 raft_consensus.cc:729] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933db276f19a490199a7496e848c03af, State: Initialized, Role: FOLLOWER
I20260620 09:21:11.026079 6567 consensus_queue.cc:260] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:11.026161 6567 ts_tablet_manager.cc:1428] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:11.027333 6570 tablet_bootstrap.cc:492] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697: No bootstrap required, opened a new log
I20260620 09:21:11.027382 6570 ts_tablet_manager.cc:1397] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20260620 09:21:11.027515 6570 raft_consensus.cc:348] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:11.027575 6570 raft_consensus.cc:374] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:11.027595 6570 raft_consensus.cc:729] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1de9b9873804459bb33266f21ec6697, State: Initialized, Role: FOLLOWER
I20260620 09:21:11.027648 6570 consensus_queue.cc:260] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:11.027738 6570 ts_tablet_manager.cc:1428] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:11.106617 6574 raft_consensus.cc:482] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:21:11.106724 6574 raft_consensus.cc:504] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:11.107050 6574 leader_election.cc:288] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 933db276f19a490199a7496e848c03af (127.4.59.1:39277), b1de9b9873804459bb33266f21ec6697 (127.4.59.3:32933)
I20260620 09:21:11.110549 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:11.107321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:57357 (local address 127.4.59.1:39277)
0620 09:21:11.107375 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:11.107377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:11.107564 (+ 187us) server_negotiation.cc:407] Connection header received
0620 09:21:11.107625 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:11.107626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:11.107637 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:11.107710 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:11.107876 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.108554 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.109849 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.110202 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.110285 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:11.110297 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:11.110447 (+ 150us) server_negotiation.cc:299] Negotiation successful
0620 09:21:11.110482 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 09:21:11.110566 6623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:11.107239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:39277 (local address 127.4.59.2:57357)
0620 09:21:11.107467 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:11.107479 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:11.107563 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:11.107716 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:11.107717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:11.107719 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:11.107796 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:11.107798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.108639 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.108640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:11.109734 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:11.109737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.110286 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.110287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:11.110370 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:11.110373 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:21:11.110442 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 09:21:11.110482 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":2,"thread_start_us":104,"threads_started":1}
I20260620 09:21:11.110778 6590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:11.107423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:33447 (local address 127.4.59.3:32933)
0620 09:21:11.107470 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:11.107473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:11.107547 (+ 74us) server_negotiation.cc:407] Connection header received
0620 09:21:11.107639 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:11.107640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:11.107648 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:11.107676 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:11.107888 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.108360 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.109842 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.110402 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.110477 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:11.110488 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:11.110674 (+ 186us) server_negotiation.cc:299] Negotiation successful
0620 09:21:11.110710 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 09:21:11.111018 6624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:11.107238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:32933 (local address 127.4.59.2:33447)
0620 09:21:11.107496 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:11.107504 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:11.107566 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:11.107755 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:11.107756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:11.107757 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:11.107808 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:11.107810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.108449 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.108450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:11.109735 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:11.109737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.110523 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.110524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:11.110597 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:11.110601 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:21:11.110891 (+ 290us) client_negotiation.cc:240] Negotiation successful
0620 09:21:11.110927 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":5,"thread_start_us":100,"threads_started":1}
I20260620 09:21:11.111093 6218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7c672c2a0f646f48f992925c200017c" candidate_uuid: "8cc7b33592094eacb715d0443a80fb9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933db276f19a490199a7496e848c03af" is_pre_election: true
I20260620 09:21:11.111181 6218 raft_consensus.cc:2454] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cc7b33592094eacb715d0443a80fb9c in term 0.
I20260620 09:21:11.111223 6498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7c672c2a0f646f48f992925c200017c" candidate_uuid: "8cc7b33592094eacb715d0443a80fb9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1de9b9873804459bb33266f21ec6697" is_pre_election: true
I20260620 09:21:11.111296 6498 raft_consensus.cc:2454] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cc7b33592094eacb715d0443a80fb9c in term 0.
I20260620 09:21:11.111367 6291 leader_election.cc:302] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cc7b33592094eacb715d0443a80fb9c, 933db276f19a490199a7496e848c03af; no voters:
I20260620 09:21:11.111475 6574 raft_consensus.cc:2781] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:21:11.111510 6574 raft_consensus.cc:482] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:21:11.111537 6574 raft_consensus.cc:3037] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:11.112183 6574 raft_consensus.cc:504] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:11.112315 6574 leader_election.cc:288] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [CANDIDATE]: Term 1 election: Requested vote from peers 933db276f19a490199a7496e848c03af (127.4.59.1:39277), b1de9b9873804459bb33266f21ec6697 (127.4.59.3:32933)
I20260620 09:21:11.112442 6218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7c672c2a0f646f48f992925c200017c" candidate_uuid: "8cc7b33592094eacb715d0443a80fb9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933db276f19a490199a7496e848c03af"
I20260620 09:21:11.112519 6218 raft_consensus.cc:3037] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:11.112529 6498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7c672c2a0f646f48f992925c200017c" candidate_uuid: "8cc7b33592094eacb715d0443a80fb9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1de9b9873804459bb33266f21ec6697"
I20260620 09:21:11.112588 6498 raft_consensus.cc:3037] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:11.113080 6498 raft_consensus.cc:2454] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cc7b33592094eacb715d0443a80fb9c in term 1.
I20260620 09:21:11.113243 6290 leader_election.cc:302] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cc7b33592094eacb715d0443a80fb9c, b1de9b9873804459bb33266f21ec6697; no voters:
I20260620 09:21:11.113314 6218 raft_consensus.cc:2454] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cc7b33592094eacb715d0443a80fb9c in term 1.
I20260620 09:21:11.113407 6574 raft_consensus.cc:2781] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:11.113545 6574 raft_consensus.cc:686] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [term 1 LEADER]: Becoming Leader. State: Replica: 8cc7b33592094eacb715d0443a80fb9c, State: Running, Role: LEADER
I20260620 09:21:11.113620 6574 consensus_queue.cc:237] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:11.114308 6060 catalog_manager.cc:5495] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8cc7b33592094eacb715d0443a80fb9c (127.4.59.2). New cstate: current_term: 1 leader_uuid: "8cc7b33592094eacb715d0443a80fb9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } health_report { overall_health: UNKNOWN } } }
I20260620 09:21:11.143245 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:11.141151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47988 (local address 127.4.59.2:36649)
0620 09:21:11.141187 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:11.141189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:11.141195 (+ 6us) server_negotiation.cc:407] Connection header received
0620 09:21:11.141221 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:11.141222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:11.141236 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:11.141289 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:21:11.141368 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.141869 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.142629 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.142815 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.142867 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:11.142977 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:21:11.143068 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:21:11.143144 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:11.143166 (+ 22us) server_negotiation.cc:299] Negotiation successful
0620 09:21:11.143181 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 09:21:11.147339 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:11.145384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:34729 (local address 127.4.59.1:39277)
0620 09:21:11.145419 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:11.145421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:11.145515 (+ 94us) server_negotiation.cc:407] Connection header received
0620 09:21:11.145563 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:11.145563 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:11.145570 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:11.145611 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:21:11.145698 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.146114 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.146798 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.146970 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.147018 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:11.147168 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:21:11.147190 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:21:11.147227 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:11.147279 (+ 52us) server_negotiation.cc:299] Negotiation successful
0620 09:21:11.147293 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 09:21:11.147554 6629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:11.145314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:39277 (local address 127.4.59.2:34729)
0620 09:21:11.145453 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:11.145463 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:11.145526 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:11.145619 (+ 93us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:11.145619 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:11.145621 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 09:21:11.145665 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:11.145666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.146176 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.146177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:11.146731 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:11.146732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.147041 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.147042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:11.147089 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:11.147101 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 09:21:11.147244 (+ 143us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 09:21:11.147245 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 09:21:11.147279 (+ 34us) client_negotiation.cc:240] Negotiation successful
0620 09:21:11.147309 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
I20260620 09:21:11.149585 6218 raft_consensus.cc:1261] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af [term 1 FOLLOWER]: Refusing update from remote peer 8cc7b33592094eacb715d0443a80fb9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:11.149637 6498 raft_consensus.cc:1261] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697 [term 1 FOLLOWER]: Refusing update from remote peer 8cc7b33592094eacb715d0443a80fb9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:11.149921 6625 consensus_queue.cc:1035] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:11.150032 6574 consensus_queue.cc:1035] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:11.156395 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:11.154343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:55237 (local address 127.4.59.2:36649)
0620 09:21:11.154437 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:11.154439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:11.154445 (+ 6us) server_negotiation.cc:407] Connection header received
0620 09:21:11.154462 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:11.154463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:11.154473 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:11.154538 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:21:11.154625 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.155052 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.155744 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.155936 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.156014 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:11.156157 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:21:11.156188 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:21:11.156241 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:11.156317 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 09:21:11.156335 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260620 09:21:11.156409 6614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:11.154266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:36649 (local address 127.4.59.1:55237)
0620 09:21:11.154319 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:11.154329 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:11.154364 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:11.154539 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:11.154540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:11.154540 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 09:21:11.154574 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:11.154576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.155128 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.155129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:11.155647 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:11.155648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:11.156016 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:11.156017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:11.156069 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:11.156075 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 09:21:11.156261 (+ 186us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 09:21:11.156263 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:21:11.156313 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 09:21:11.156353 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 09:21:11.156731 6597 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 09:21:12.677029 6061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:12.677130 6061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:12.678013 6061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e: Sending DeleteTablet for 3 replicas of tablet e7c672c2a0f646f48f992925c200017c
I20260620 09:21:12.678298 6198 tablet_service.cc:1514] Processing DeleteTablet for tablet e7c672c2a0f646f48f992925c200017c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:12 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:35802
I20260620 09:21:12.678337 6338 tablet_service.cc:1514] Processing DeleteTablet for tablet e7c672c2a0f646f48f992925c200017c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:12 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:47972
I20260620 09:21:12.678328 6477 tablet_service.cc:1514] Processing DeleteTablet for tablet e7c672c2a0f646f48f992925c200017c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:12 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:42662
I20260620 09:21:12.678555 6655 tablet_replica.cc:330] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c: stopping tablet replica
I20260620 09:21:12.679831 6655 raft_consensus.cc:2229] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:21:12.680032 6655 raft_consensus.cc:2258] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:12.680349 6655 ts_tablet_manager.cc:1905] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:12.681262 6657 tablet_replica.cc:330] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697: stopping tablet replica
I20260620 09:21:12.681252 6656 tablet_replica.cc:330] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af: stopping tablet replica
I20260620 09:21:12.681322 6656 raft_consensus.cc:2229] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:12.681337 6657 raft_consensus.cc:2229] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:12.681375 6657 raft_consensus.cc:2258] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:12.681546 6656 raft_consensus.cc:2258] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:12.681581 6657 ts_tablet_manager.cc:1905] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:12.681823 6656 ts_tablet_manager.cc:1905] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:12.682304 6655 ts_tablet_manager.cc:1918] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 09:21:12.682367 6655 log.cc:1196] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/e7c672c2a0f646f48f992925c200017c
I20260620 09:21:12.682628 6655 ts_tablet_manager.cc:1939] T e7c672c2a0f646f48f992925c200017c P 8cc7b33592094eacb715d0443a80fb9c: Deleting consensus metadata
I20260620 09:21:12.682961 6657 ts_tablet_manager.cc:1918] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 09:21:12.683037 6657 log.cc:1196] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/e7c672c2a0f646f48f992925c200017c
I20260620 09:21:12.683251 6657 ts_tablet_manager.cc:1939] T e7c672c2a0f646f48f992925c200017c P b1de9b9873804459bb33266f21ec6697: Deleting consensus metadata
I20260620 09:21:12.683588 6046 catalog_manager.cc:4841] TS b1de9b9873804459bb33266f21ec6697 (127.4.59.3:32933): tablet e7c672c2a0f646f48f992925c200017c (table test-table [id=1697c7312dcb47f5a22d961dc361385f]) successfully deleted
I20260620 09:21:12.683604 6656 ts_tablet_manager.cc:1918] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 09:21:12.683656 6656 log.cc:1196] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/wal/wals/e7c672c2a0f646f48f992925c200017c
I20260620 09:21:12.683853 6656 ts_tablet_manager.cc:1939] T e7c672c2a0f646f48f992925c200017c P 933db276f19a490199a7496e848c03af: Deleting consensus metadata
I20260620 09:21:12.684213 6047 catalog_manager.cc:4841] TS 933db276f19a490199a7496e848c03af (127.4.59.1:39277): tablet e7c672c2a0f646f48f992925c200017c (table test-table [id=1697c7312dcb47f5a22d961dc361385f]) successfully deleted
I20260620 09:21:12.685439 6047 catalog_manager.cc:4841] TS 8cc7b33592094eacb715d0443a80fb9c (127.4.59.2:36649): tablet e7c672c2a0f646f48f992925c200017c (table test-table [id=1697c7312dcb47f5a22d961dc361385f]) successfully deleted
Jun 20 09:21:12 dist-test-slave-414w krb5kdc[6016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947272, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 09:21:12 dist-test-slave-414w krb5kdc[6016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947272, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:12.708227 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:12.697092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45568 (local address 127.4.59.62:40191)
0620 09:21:12.697250 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:12.697252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:12.697262 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:21:12.697338 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:12.697340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:12.697353 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:12.697423 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:12.697975 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:12.698518 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:12.699291 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:12.699497 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:12.699581 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:12.705663 (+ 6082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:12.705664 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:12.705665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:12.705673 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:12.706976 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:12.707434 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:12.707435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:12.707436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:12.707496 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:12.707695 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:12.707696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:12.707696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:12.707818 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:12.707920 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:12.708034 (+ 114us) server_negotiation.cc:299] Negotiation successful
0620 09:21:12.708116 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260620 09:21:12.709611 6061 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:21:12.709760 6061 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:21:12.711360 6198 tablet_service.cc:1467] Processing CreateTablet for tablet bc4b5cc2040b401b8f7e35ad675f9709 (DEFAULT_TABLE table=test-table [id=5584f85bb2804b138e16f68c83899cc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:12.711371 6338 tablet_service.cc:1467] Processing CreateTablet for tablet bc4b5cc2040b401b8f7e35ad675f9709 (DEFAULT_TABLE table=test-table [id=5584f85bb2804b138e16f68c83899cc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:12.711457 6477 tablet_service.cc:1467] Processing CreateTablet for tablet bc4b5cc2040b401b8f7e35ad675f9709 (DEFAULT_TABLE table=test-table [id=5584f85bb2804b138e16f68c83899cc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:12.711498 6198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc4b5cc2040b401b8f7e35ad675f9709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:12.711504 6338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc4b5cc2040b401b8f7e35ad675f9709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:12.711586 6477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc4b5cc2040b401b8f7e35ad675f9709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:12.712913 6666 tablet_bootstrap.cc:492] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c: Bootstrap starting.
I20260620 09:21:12.713040 6668 tablet_bootstrap.cc:492] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697: Bootstrap starting.
I20260620 09:21:12.713217 6667 tablet_bootstrap.cc:492] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af: Bootstrap starting.
I20260620 09:21:12.713564 6667 tablet_bootstrap.cc:654] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:12.713553 6666 tablet_bootstrap.cc:654] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:12.713578 6668 tablet_bootstrap.cc:654] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:12.714161 6667 tablet_bootstrap.cc:492] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af: No bootstrap required, opened a new log
I20260620 09:21:12.714156 6666 tablet_bootstrap.cc:492] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c: No bootstrap required, opened a new log
I20260620 09:21:12.714156 6668 tablet_bootstrap.cc:492] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697: No bootstrap required, opened a new log
I20260620 09:21:12.714226 6668 ts_tablet_manager.cc:1397] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:12.714226 6666 ts_tablet_manager.cc:1397] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 09:21:12.714226 6667 ts_tablet_manager.cc:1397] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:12.714347 6667 raft_consensus.cc:348] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:12.714342 6668 raft_consensus.cc:348] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:12.714342 6666 raft_consensus.cc:348] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:12.714398 6667 raft_consensus.cc:374] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:12.714412 6667 raft_consensus.cc:729] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933db276f19a490199a7496e848c03af, State: Initialized, Role: FOLLOWER
I20260620 09:21:12.714406 6666 raft_consensus.cc:374] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:12.714406 6668 raft_consensus.cc:374] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:12.714430 6666 raft_consensus.cc:729] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc7b33592094eacb715d0443a80fb9c, State: Initialized, Role: FOLLOWER
I20260620 09:21:12.714447 6668 raft_consensus.cc:729] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1de9b9873804459bb33266f21ec6697, State: Initialized, Role: FOLLOWER
I20260620 09:21:12.714469 6667 consensus_queue.cc:260] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:12.714489 6666 consensus_queue.cc:260] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:12.714491 6668 consensus_queue.cc:260] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:12.714574 6667 ts_tablet_manager.cc:1428] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:12.714596 6666 ts_tablet_manager.cc:1428] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260620 09:21:12.714656 6668 ts_tablet_manager.cc:1428] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:12.971472 6672 raft_consensus.cc:482] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:21:12.971570 6672 raft_consensus.cc:504] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:12.971876 6672 leader_election.cc:288] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 933db276f19a490199a7496e848c03af (127.4.59.1:39277), 8cc7b33592094eacb715d0443a80fb9c (127.4.59.2:36649)
I20260620 09:21:12.972127 6358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc4b5cc2040b401b8f7e35ad675f9709" candidate_uuid: "b1de9b9873804459bb33266f21ec6697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cc7b33592094eacb715d0443a80fb9c" is_pre_election: true
I20260620 09:21:12.972131 6218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc4b5cc2040b401b8f7e35ad675f9709" candidate_uuid: "b1de9b9873804459bb33266f21ec6697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933db276f19a490199a7496e848c03af" is_pre_election: true
I20260620 09:21:12.972239 6218 raft_consensus.cc:2454] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1de9b9873804459bb33266f21ec6697 in term 0.
I20260620 09:21:12.972241 6358 raft_consensus.cc:2454] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1de9b9873804459bb33266f21ec6697 in term 0.
I20260620 09:21:12.972419 6431 leader_election.cc:302] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cc7b33592094eacb715d0443a80fb9c, b1de9b9873804459bb33266f21ec6697; no voters:
I20260620 09:21:12.972555 6672 raft_consensus.cc:2781] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:21:12.972610 6672 raft_consensus.cc:482] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:21:12.972625 6672 raft_consensus.cc:3037] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:12.973343 6672 raft_consensus.cc:504] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:12.973464 6672 leader_election.cc:288] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [CANDIDATE]: Term 1 election: Requested vote from peers 933db276f19a490199a7496e848c03af (127.4.59.1:39277), 8cc7b33592094eacb715d0443a80fb9c (127.4.59.2:36649)
I20260620 09:21:12.973634 6218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc4b5cc2040b401b8f7e35ad675f9709" candidate_uuid: "b1de9b9873804459bb33266f21ec6697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933db276f19a490199a7496e848c03af"
I20260620 09:21:12.973639 6358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc4b5cc2040b401b8f7e35ad675f9709" candidate_uuid: "b1de9b9873804459bb33266f21ec6697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cc7b33592094eacb715d0443a80fb9c"
I20260620 09:21:12.973697 6218 raft_consensus.cc:3037] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:12.973721 6358 raft_consensus.cc:3037] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:12.974278 6218 raft_consensus.cc:2454] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1de9b9873804459bb33266f21ec6697 in term 1.
I20260620 09:21:12.974354 6358 raft_consensus.cc:2454] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1de9b9873804459bb33266f21ec6697 in term 1.
I20260620 09:21:12.974442 6431 leader_election.cc:302] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 933db276f19a490199a7496e848c03af, b1de9b9873804459bb33266f21ec6697; no voters:
I20260620 09:21:12.974551 6672 raft_consensus.cc:2781] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:12.974737 6672 raft_consensus.cc:686] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [term 1 LEADER]: Becoming Leader. State: Replica: b1de9b9873804459bb33266f21ec6697, State: Running, Role: LEADER
I20260620 09:21:12.974788 6672 consensus_queue.cc:237] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:12.975394 6061 catalog_manager.cc:5495] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1de9b9873804459bb33266f21ec6697 (127.4.59.3). New cstate: current_term: 1 leader_uuid: "b1de9b9873804459bb33266f21ec6697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } health_report { overall_health: HEALTHY } } }
I20260620 09:21:13.000548 6676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:12.998194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42680 (local address 127.4.59.3:32933)
0620 09:21:12.998323 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:12.998325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:12.998333 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:21:12.998362 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:12.998363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:12.998372 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:12.998457 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:21:12.998584 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:12.999065 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:12.999815 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:13.000028 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:13.000110 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:13.000265 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:21:13.000298 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:21:13.000380 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:13.000442 (+ 62us) server_negotiation.cc:299] Negotiation successful
0620 09:21:13.000462 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":79,"threads_started":1}
I20260620 09:21:13.002863 6358 raft_consensus.cc:1261] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c [term 1 FOLLOWER]: Refusing update from remote peer b1de9b9873804459bb33266f21ec6697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:13.002900 6218 raft_consensus.cc:1261] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af [term 1 FOLLOWER]: Refusing update from remote peer b1de9b9873804459bb33266f21ec6697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:13.003077 6672 consensus_queue.cc:1035] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:13.003173 6673 consensus_queue.cc:1035] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:13.012748 6061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:13.012827 6061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:13.013448 6061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e: Sending DeleteTablet for 3 replicas of tablet bc4b5cc2040b401b8f7e35ad675f9709
I20260620 09:21:13.013667 6198 tablet_service.cc:1514] Processing DeleteTablet for tablet bc4b5cc2040b401b8f7e35ad675f9709 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:13 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:35802
I20260620 09:21:13.013692 6477 tablet_service.cc:1514] Processing DeleteTablet for tablet bc4b5cc2040b401b8f7e35ad675f9709 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:13 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:42662
I20260620 09:21:13.013692 6338 tablet_service.cc:1514] Processing DeleteTablet for tablet bc4b5cc2040b401b8f7e35ad675f9709 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:13 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:47972
I20260620 09:21:13.013761 6656 tablet_replica.cc:330] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af: stopping tablet replica
I20260620 09:21:13.013772 6657 tablet_replica.cc:330] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697: stopping tablet replica
I20260620 09:21:13.013772 6655 tablet_replica.cc:330] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c: stopping tablet replica
I20260620 09:21:13.013813 6656 raft_consensus.cc:2229] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:13.013815 6655 raft_consensus.cc:2229] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:13.013820 6657 raft_consensus.cc:2229] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:21:13.013850 6655 raft_consensus.cc:2258] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:13.013849 6656 raft_consensus.cc:2258] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:13.013906 6657 raft_consensus.cc:2258] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:13.014178 6657 ts_tablet_manager.cc:1905] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:13.014222 6656 ts_tablet_manager.cc:1905] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:13.015661 6656 ts_tablet_manager.cc:1918] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 09:21:13.015736 6656 log.cc:1196] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-0/wal/wals/bc4b5cc2040b401b8f7e35ad675f9709
I20260620 09:21:13.015753 6657 ts_tablet_manager.cc:1918] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 09:21:13.015801 6657 log.cc:1196] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/bc4b5cc2040b401b8f7e35ad675f9709
I20260620 09:21:13.015911 6656 ts_tablet_manager.cc:1939] T bc4b5cc2040b401b8f7e35ad675f9709 P 933db276f19a490199a7496e848c03af: Deleting consensus metadata
I20260620 09:21:13.016011 6657 ts_tablet_manager.cc:1939] T bc4b5cc2040b401b8f7e35ad675f9709 P b1de9b9873804459bb33266f21ec6697: Deleting consensus metadata
I20260620 09:21:13.016167 6047 catalog_manager.cc:4841] TS 933db276f19a490199a7496e848c03af (127.4.59.1:39277): tablet bc4b5cc2040b401b8f7e35ad675f9709 (table test-table [id=5584f85bb2804b138e16f68c83899cc6]) successfully deleted
I20260620 09:21:13.016250 6046 catalog_manager.cc:4841] TS b1de9b9873804459bb33266f21ec6697 (127.4.59.3:32933): tablet bc4b5cc2040b401b8f7e35ad675f9709 (table test-table [id=5584f85bb2804b138e16f68c83899cc6]) successfully deleted
I20260620 09:21:13.016435 6655 ts_tablet_manager.cc:1905] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:13.017052 6061 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:21:13.017248 6061 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:21:13.017903 6655 ts_tablet_manager.cc:1918] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 09:21:13.017961 6655 log.cc:1196] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/bc4b5cc2040b401b8f7e35ad675f9709
I20260620 09:21:13.018205 6655 ts_tablet_manager.cc:1939] T bc4b5cc2040b401b8f7e35ad675f9709 P 8cc7b33592094eacb715d0443a80fb9c: Deleting consensus metadata
I20260620 09:21:13.018548 6198 tablet_service.cc:1467] Processing CreateTablet for tablet 4f083148a430490f9d5149869e3c8504 (DEFAULT_TABLE table=test-table [id=4ed3cf9e119c449186f766635c5123d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:13.018559 6477 tablet_service.cc:1467] Processing CreateTablet for tablet 4f083148a430490f9d5149869e3c8504 (DEFAULT_TABLE table=test-table [id=4ed3cf9e119c449186f766635c5123d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:13.018585 6338 tablet_service.cc:1467] Processing CreateTablet for tablet 4f083148a430490f9d5149869e3c8504 (DEFAULT_TABLE table=test-table [id=4ed3cf9e119c449186f766635c5123d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:13.018676 6477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f083148a430490f9d5149869e3c8504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:13.018676 6338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f083148a430490f9d5149869e3c8504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:13.018683 6198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f083148a430490f9d5149869e3c8504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:13.019632 6667 tablet_bootstrap.cc:492] T 4f083148a430490f9d5149869e3c8504 P 933db276f19a490199a7496e848c03af: Bootstrap starting.
I20260620 09:21:13.019826 6668 tablet_bootstrap.cc:492] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697: Bootstrap starting.
I20260620 09:21:13.019837 6666 tablet_bootstrap.cc:492] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c: Bootstrap starting.
I20260620 09:21:13.019860 6047 catalog_manager.cc:4841] TS 8cc7b33592094eacb715d0443a80fb9c (127.4.59.2:36649): tablet bc4b5cc2040b401b8f7e35ad675f9709 (table test-table [id=5584f85bb2804b138e16f68c83899cc6]) successfully deleted
I20260620 09:21:13.019979 6667 tablet_bootstrap.cc:654] T 4f083148a430490f9d5149869e3c8504 P 933db276f19a490199a7496e848c03af: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:13.020140 6666 tablet_bootstrap.cc:654] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:13.020159 6668 tablet_bootstrap.cc:654] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:13.020538 6667 tablet_bootstrap.cc:492] T 4f083148a430490f9d5149869e3c8504 P 933db276f19a490199a7496e848c03af: No bootstrap required, opened a new log
I20260620 09:21:13.020586 6667 ts_tablet_manager.cc:1397] T 4f083148a430490f9d5149869e3c8504 P 933db276f19a490199a7496e848c03af: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:13.020696 6668 tablet_bootstrap.cc:492] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697: No bootstrap required, opened a new log
I20260620 09:21:13.020709 6667 raft_consensus.cc:348] T 4f083148a430490f9d5149869e3c8504 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:13.020753 6668 ts_tablet_manager.cc:1397] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:13.020773 6667 raft_consensus.cc:374] T 4f083148a430490f9d5149869e3c8504 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:13.020793 6667 raft_consensus.cc:729] T 4f083148a430490f9d5149869e3c8504 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933db276f19a490199a7496e848c03af, State: Initialized, Role: FOLLOWER
I20260620 09:21:13.020860 6667 consensus_queue.cc:260] T 4f083148a430490f9d5149869e3c8504 P 933db276f19a490199a7496e848c03af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:13.020874 6668 raft_consensus.cc:348] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:13.020939 6668 raft_consensus.cc:374] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:13.020949 6667 ts_tablet_manager.cc:1428] T 4f083148a430490f9d5149869e3c8504 P 933db276f19a490199a7496e848c03af: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:13.020960 6668 raft_consensus.cc:729] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1de9b9873804459bb33266f21ec6697, State: Initialized, Role: FOLLOWER
I20260620 09:21:13.021003 6668 consensus_queue.cc:260] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:13.021088 6668 ts_tablet_manager.cc:1428] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:13.021258 6666 tablet_bootstrap.cc:492] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c: No bootstrap required, opened a new log
I20260620 09:21:13.021309 6666 ts_tablet_manager.cc:1397] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 09:21:13.021450 6666 raft_consensus.cc:348] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:13.021528 6666 raft_consensus.cc:374] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:13.021548 6666 raft_consensus.cc:729] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc7b33592094eacb715d0443a80fb9c, State: Initialized, Role: FOLLOWER
I20260620 09:21:13.021587 6666 consensus_queue.cc:260] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:13.021679 6666 ts_tablet_manager.cc:1428] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:13.062180 6649 raft_consensus.cc:482] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:21:13.062271 6649 raft_consensus.cc:504] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:13.062438 6649 leader_election.cc:288] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 933db276f19a490199a7496e848c03af (127.4.59.1:39277), b1de9b9873804459bb33266f21ec6697 (127.4.59.3:32933)
I20260620 09:21:13.062623 6218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f083148a430490f9d5149869e3c8504" candidate_uuid: "8cc7b33592094eacb715d0443a80fb9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933db276f19a490199a7496e848c03af" is_pre_election: true
I20260620 09:21:13.062659 6498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f083148a430490f9d5149869e3c8504" candidate_uuid: "8cc7b33592094eacb715d0443a80fb9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1de9b9873804459bb33266f21ec6697" is_pre_election: true
I20260620 09:21:13.062734 6498 raft_consensus.cc:2454] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cc7b33592094eacb715d0443a80fb9c in term 0.
I20260620 09:21:13.062732 6218 raft_consensus.cc:2454] T 4f083148a430490f9d5149869e3c8504 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cc7b33592094eacb715d0443a80fb9c in term 0.
I20260620 09:21:13.062904 6291 leader_election.cc:302] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cc7b33592094eacb715d0443a80fb9c, 933db276f19a490199a7496e848c03af; no voters:
I20260620 09:21:13.063028 6649 raft_consensus.cc:2781] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:21:13.063076 6649 raft_consensus.cc:482] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:21:13.063090 6649 raft_consensus.cc:3037] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:13.063735 6649 raft_consensus.cc:504] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:13.063881 6649 leader_election.cc:288] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [CANDIDATE]: Term 1 election: Requested vote from peers 933db276f19a490199a7496e848c03af (127.4.59.1:39277), b1de9b9873804459bb33266f21ec6697 (127.4.59.3:32933)
I20260620 09:21:13.064010 6218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f083148a430490f9d5149869e3c8504" candidate_uuid: "8cc7b33592094eacb715d0443a80fb9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933db276f19a490199a7496e848c03af"
I20260620 09:21:13.064043 6498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f083148a430490f9d5149869e3c8504" candidate_uuid: "8cc7b33592094eacb715d0443a80fb9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1de9b9873804459bb33266f21ec6697"
I20260620 09:21:13.064100 6218 raft_consensus.cc:3037] T 4f083148a430490f9d5149869e3c8504 P 933db276f19a490199a7496e848c03af [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:13.064124 6498 raft_consensus.cc:3037] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:13.064805 6498 raft_consensus.cc:2454] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cc7b33592094eacb715d0443a80fb9c in term 1.
I20260620 09:21:13.064805 6218 raft_consensus.cc:2454] T 4f083148a430490f9d5149869e3c8504 P 933db276f19a490199a7496e848c03af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cc7b33592094eacb715d0443a80fb9c in term 1.
I20260620 09:21:13.064985 6291 leader_election.cc:302] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cc7b33592094eacb715d0443a80fb9c, 933db276f19a490199a7496e848c03af; no voters:
I20260620 09:21:13.065086 6649 raft_consensus.cc:2781] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:13.065196 6649 raft_consensus.cc:686] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [term 1 LEADER]: Becoming Leader. State: Replica: 8cc7b33592094eacb715d0443a80fb9c, State: Running, Role: LEADER
I20260620 09:21:13.065253 6649 consensus_queue.cc:237] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } }
I20260620 09:21:13.065938 6061 catalog_manager.cc:5495] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8cc7b33592094eacb715d0443a80fb9c (127.4.59.2). New cstate: current_term: 1 leader_uuid: "8cc7b33592094eacb715d0443a80fb9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cc7b33592094eacb715d0443a80fb9c" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 36649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 } health_report { overall_health: UNKNOWN } } }
I20260620 09:21:13.083698 6685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:13.081554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47994 (local address 127.4.59.2:36649)
0620 09:21:13.081725 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:13.081727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:13.081735 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:21:13.081756 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:13.081756 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:13.081767 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:13.081851 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:21:13.082021 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:13.082501 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:13.083175 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:13.083364 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:13.083415 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:13.083526 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:21:13.083549 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:21:13.083583 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:13.083630 (+ 47us) server_negotiation.cc:299] Negotiation successful
0620 09:21:13.083643 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":109,"threads_started":1}
I20260620 09:21:13.085780 6218 raft_consensus.cc:1261] T 4f083148a430490f9d5149869e3c8504 P 933db276f19a490199a7496e848c03af [term 1 FOLLOWER]: Refusing update from remote peer 8cc7b33592094eacb715d0443a80fb9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:13.085800 6498 raft_consensus.cc:1261] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697 [term 1 FOLLOWER]: Refusing update from remote peer 8cc7b33592094eacb715d0443a80fb9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:13.085971 6574 consensus_queue.cc:1035] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "933db276f19a490199a7496e848c03af" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:13.086076 6649 consensus_queue.cc:1035] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1de9b9873804459bb33266f21ec6697" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 32933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 09:21:13.090299 6637 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 09:21:14.611680 6061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:14.611784 6061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:14.612705 6061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81f5c4b666834bd9acd7cb6be12fd24e: Sending DeleteTablet for 3 replicas of tablet 4f083148a430490f9d5149869e3c8504
I20260620 09:21:14.612967 6198 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f083148a430490f9d5149869e3c8504 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:14 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:35802
I20260620 09:21:14.612998 6338 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f083148a430490f9d5149869e3c8504 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:14 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:47972
I20260620 09:21:14.613086 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 6133
I20260620 09:21:14.613174 6477 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f083148a430490f9d5149869e3c8504 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:14 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:42662
I20260620 09:21:14.614609 6712 tablet_replica.cc:330] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697: stopping tablet replica
I20260620 09:21:14.614673 6712 raft_consensus.cc:2229] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:14.614701 6712 raft_consensus.cc:2258] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:14.614974 6712 ts_tablet_manager.cc:1905] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:14.616923 6712 ts_tablet_manager.cc:1918] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 09:21:14.616990 6712 log.cc:1196] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/4f083148a430490f9d5149869e3c8504
I20260620 09:21:14.617237 6712 ts_tablet_manager.cc:1939] T 4f083148a430490f9d5149869e3c8504 P b1de9b9873804459bb33266f21ec6697: Deleting consensus metadata
I20260620 09:21:14.617247 6711 tablet_replica.cc:330] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c: stopping tablet replica
I20260620 09:21:14.617326 6711 raft_consensus.cc:2229] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:21:14.617398 6711 raft_consensus.cc:2258] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:14.617630 6711 ts_tablet_manager.cc:1905] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:14.617897 6046 catalog_manager.cc:4841] TS b1de9b9873804459bb33266f21ec6697 (127.4.59.3:32933): tablet 4f083148a430490f9d5149869e3c8504 (table test-table [id=4ed3cf9e119c449186f766635c5123d0]) successfully deleted
I20260620 09:21:14.619151 6711 ts_tablet_manager.cc:1918] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 09:21:14.619204 6711 log.cc:1196] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/4f083148a430490f9d5149869e3c8504
I20260620 09:21:14.619457 6711 ts_tablet_manager.cc:1939] T 4f083148a430490f9d5149869e3c8504 P 8cc7b33592094eacb715d0443a80fb9c: Deleting consensus metadata
I20260620 09:21:14.619764 6047 catalog_manager.cc:4841] TS 8cc7b33592094eacb715d0443a80fb9c (127.4.59.2:36649): tablet 4f083148a430490f9d5149869e3c8504 (table test-table [id=4ed3cf9e119c449186f766635c5123d0]) successfully deleted
W20260620 09:21:14.619927 6047 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.59.1:39277 (error 108)
I20260620 09:21:14.620177 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 6273
I20260620 09:21:14.621986 6714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:14.620325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:39277 (local address 127.0.0.1:35812)
0620 09:21:14.621906 (+ 1581us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:14.621954 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:39277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1554,"thread_start_us":62,"threads_started":1}
W20260620 09:21:14.622133 6047 catalog_manager.cc:4584] TS 933db276f19a490199a7496e848c03af (127.4.59.1:39277): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f083148a430490f9d5149869e3c8504: Network error: Client connection negotiation failed: client connection to 127.4.59.1:39277: connect: Connection refused (error 111)
I20260620 09:21:14.626013 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 6413
I20260620 09:21:14.632149 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 6029
2026-06-20T09:21:14Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6128 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:21:14 dist-test-slave-414w krb5kdc[6717](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:21:14 dist-test-slave-414w krb5kdc[6717](info): set up 2 sockets
Jun 20 09:21:14 dist-test-slave-414w krb5kdc[6717](info): commencing operation
krb5kdc: starting...
W20260620 09:21:15.681454 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:21:15 dist-test-slave-414w krb5kdc[6717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:21:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:21:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:21:15.796339 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:40441
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:40441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:15.877370 6730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:15.877576 6730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:15.877610 6730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:15.878736 6730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:15.878801 6730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:15.878825 6730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:15.878847 6730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:15.878873 6730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:21:15.880281 6730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:40441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:40441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:15.880542 6730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:15.880754 6730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:15.885962 6736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:15.886111 6735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:15.885977 6738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:15.886158 6730 server_base.cc:1034] running on GCE node
I20260620 09:21:15.886626 6730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:15.886920 6730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:15.888068 6730 hybrid_clock.cc:648] HybridClock initialized: now 1781947275888046 us; error 36 us; skew 500 ppm
Jun 20 09:21:15 dist-test-slave-414w krb5kdc[6717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947275, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:15.890714 6730 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:21:15.891233 6730 webserver.cc:460] Webserver started at http://127.4.59.62:43391/ using document root <none> and password file <none>
I20260620 09:21:15.891462 6730 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:15.891507 6730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:15.891666 6730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:15.892727 6730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "67b6710ceb2744768043d506b3725aa0"
format_stamp: "Formatted at 2026-06-20 09:21:15 on dist-test-slave-414w"
server_key: "f7fb0ed20a25e7c161b7584c4b42ce77"
server_key_iv: "f21239deb19827e1784006d5fa94aa03"
server_key_version: "encryptionkey@0"
I20260620 09:21:15.893086 6730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "67b6710ceb2744768043d506b3725aa0"
format_stamp: "Formatted at 2026-06-20 09:21:15 on dist-test-slave-414w"
server_key: "f7fb0ed20a25e7c161b7584c4b42ce77"
server_key_iv: "f21239deb19827e1784006d5fa94aa03"
server_key_version: "encryptionkey@0"
I20260620 09:21:15.894457 6730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:15.895296 6745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:15.895538 6730 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:15.895649 6730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "67b6710ceb2744768043d506b3725aa0"
format_stamp: "Formatted at 2026-06-20 09:21:15 on dist-test-slave-414w"
server_key: "f7fb0ed20a25e7c161b7584c4b42ce77"
server_key_iv: "f21239deb19827e1784006d5fa94aa03"
server_key_version: "encryptionkey@0"
I20260620 09:21:15.895740 6730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:15.908777 6730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:15.911087 6730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:15.911253 6730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:15.915257 6730 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:40441
I20260620 09:21:15.915287 6797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:40441 every 8 connection(s)
I20260620 09:21:15.915637 6730 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:21:15.916501 6798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:15.916836 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 6730
I20260620 09:21:15.916908 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:21:15.917097 4332 external_mini_cluster.cc:1428] Setting key ddd124f8200fcdeb4b9d72666168e45d
I20260620 09:21:15.919617 6798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0: Bootstrap starting.
I20260620 09:21:15.920306 6798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:15.920580 6798 log.cc:824] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:15.921311 6798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0: No bootstrap required, opened a new log
I20260620 09:21:15.923612 6798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b6710ceb2744768043d506b3725aa0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40441 } }
I20260620 09:21:15.923815 6798 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:15.923861 6798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67b6710ceb2744768043d506b3725aa0, State: Initialized, Role: FOLLOWER
I20260620 09:21:15.923980 6798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b6710ceb2744768043d506b3725aa0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40441 } }
I20260620 09:21:15.924065 6798 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:21:15.924098 6798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:21:15.924153 6798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [term 0 FOLLOWER]: Advancing to term 1
Jun 20 09:21:15 dist-test-slave-414w krb5kdc[6717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:15.924868 6798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b6710ceb2744768043d506b3725aa0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40441 } }
I20260620 09:21:15.924993 6798 leader_election.cc:302] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67b6710ceb2744768043d506b3725aa0; no voters:
I20260620 09:21:15.925163 6798 leader_election.cc:288] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:21:15.925191 6803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:15.925355 6803 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [term 1 LEADER]: Becoming Leader. State: Replica: 67b6710ceb2744768043d506b3725aa0, State: Running, Role: LEADER
I20260620 09:21:15.925434 6798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:15.925522 6803 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b6710ceb2744768043d506b3725aa0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40441 } }
I20260620 09:21:15.925932 6805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67b6710ceb2744768043d506b3725aa0. Latest consensus state: current_term: 1 leader_uuid: "67b6710ceb2744768043d506b3725aa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b6710ceb2744768043d506b3725aa0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40441 } } }
I20260620 09:21:15.925925 6804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67b6710ceb2744768043d506b3725aa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b6710ceb2744768043d506b3725aa0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40441 } } }
I20260620 09:21:15.926560 6804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:15.926113 6805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0 [sys.catalog]: This master's current role is: LEADER
W20260620 09:21:15.927309 6818 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:21:15.927362 6818 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:21:15.927826 6820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:21:15.928020 6820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:21:15.929304 6801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:15.917644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54518 (local address 127.4.59.62:40441)
0620 09:21:15.917887 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:15.917897 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:15.917929 (+ 32us) server_negotiation.cc:407] Connection header received
0620 09:21:15.918445 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:15.918454 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:15.918514 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:15.918676 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:15.919288 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:15.920623 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:15.921600 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:15.922205 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:15.922318 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:15.924896 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:15.924900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:15.924900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:15.924910 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:15.927625 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:15.928001 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:15.928004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:15.928005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:15.928113 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:15.928324 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:15.928325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:15.928325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:15.928628 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:15.928789 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:15.928887 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 09:21:15.929027 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":120,"threads_started":1}
I20260620 09:21:15.929663 6820 catalog_manager.cc:1347] Generated new cluster ID: 3cf132a97d6246cd8214c187c6174e26
I20260620 09:21:15.929711 6820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:21:15.956786 6820 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:21:15.957522 6820 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:21:15.965075 6820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67b6710ceb2744768043d506b3725aa0: Generated new TSK 0
I20260620 09:21:15.965337 6820 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:21:15.992975 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:40441
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:16.073817 6824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:16.074004 6824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:16.074026 6824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:16.075100 6824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:16.075135 6824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:16.075161 6824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:21:16.076498 6824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:40441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:16.076676 6824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:16.076850 6824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:16.081723 6832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:16.081735 6829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:16.081732 6830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:16.081812 6824 server_base.cc:1034] running on GCE node
I20260620 09:21:16.082221 6824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:16.082469 6824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:16.083630 6824 hybrid_clock.cc:648] HybridClock initialized: now 1781947276083629 us; error 37 us; skew 500 ppm
Jun 20 09:21:16 dist-test-slave-414w krb5kdc[6717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947276, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:16.086225 6824 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:21:16.086588 6824 webserver.cc:460] Webserver started at http://127.4.59.1:34841/ using document root <none> and password file <none>
I20260620 09:21:16.086774 6824 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:16.086823 6824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:16.086912 6824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:16.087857 6824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "6334af0dbdfb4c4cb6afa3a3c5ae19ff"
format_stamp: "Formatted at 2026-06-20 09:21:16 on dist-test-slave-414w"
server_key: "7ef3ca0d8dae2069124ea66e0d3ce362"
server_key_iv: "2feee77808bb413970a52ab5e407f791"
server_key_version: "encryptionkey@0"
I20260620 09:21:16.088141 6824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "6334af0dbdfb4c4cb6afa3a3c5ae19ff"
format_stamp: "Formatted at 2026-06-20 09:21:16 on dist-test-slave-414w"
server_key: "7ef3ca0d8dae2069124ea66e0d3ce362"
server_key_iv: "2feee77808bb413970a52ab5e407f791"
server_key_version: "encryptionkey@0"
I20260620 09:21:16.089403 6824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:21:16.090109 6839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:16.090281 6824 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:16.090351 6824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "6334af0dbdfb4c4cb6afa3a3c5ae19ff"
format_stamp: "Formatted at 2026-06-20 09:21:16 on dist-test-slave-414w"
server_key: "7ef3ca0d8dae2069124ea66e0d3ce362"
server_key_iv: "2feee77808bb413970a52ab5e407f791"
server_key_version: "encryptionkey@0"
I20260620 09:21:16.090458 6824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:16.102854 6824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:16.105216 6824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:16.105376 6824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:16.105610 6824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:16.105923 6824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:16.105978 6824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:16.106019 6824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:16.106045 6824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:16.111629 6824 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:34671
I20260620 09:21:16.111704 6952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:34671 every 8 connection(s)
I20260620 09:21:16.112016 6824 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:21:16.117877 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 6824
I20260620 09:21:16.117950 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:21:16.118121 4332 external_mini_cluster.cc:1428] Setting key 54d9e027a7840a4338648c442716c948
Jun 20 09:21:16 dist-test-slave-414w krb5kdc[6717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947276, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:16.121697 6801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:16.112788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:39587 (local address 127.4.59.62:40441)
0620 09:21:16.112879 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:16.112881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:16.113061 (+ 180us) server_negotiation.cc:407] Connection header received
0620 09:21:16.113638 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:16.113639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:16.113651 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:16.113711 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:16.114647 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:16.115152 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:16.116183 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:16.116398 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:16.116483 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:16.118756 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:16.118757 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:16.118757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:16.118764 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:16.120185 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:16.120695 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:16.120696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:16.120696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:16.120744 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:16.121024 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:16.121025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:16.121025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:16.121121 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:16.121255 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:16.121560 (+ 305us) server_negotiation.cc:299] Negotiation successful
0620 09:21:16.121634 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 09:21:16.121985 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:16.112763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40441 (local address 127.4.59.1:39587)
0620 09:21:16.112936 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:16.112979 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:16.113531 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:16.113779 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:16.113783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:16.114356 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:16.114543 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:16.114547 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:16.115273 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:16.115274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:16.116074 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:16.116076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:16.116495 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:16.116496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:16.116625 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:16.117115 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:16.117124 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:16.118647 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:16.120267 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:16.120292 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:16.120294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:16.120615 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:16.120811 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:16.120811 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:16.120812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:16.120955 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:16.121247 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:16.121254 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:16.121426 (+ 172us) client_negotiation.cc:769] Sending connection context
0620 09:21:16.121542 (+ 116us) client_negotiation.cc:240] Negotiation successful
0620 09:21:16.121713 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260620 09:21:16.122615 6953 heartbeater.cc:347] Connected to a master server at 127.4.59.62:40441
I20260620 09:21:16.122725 6953 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:16.122915 6953 heartbeater.cc:511] Master 127.4.59.62:40441 requested a full tablet report, sending...
I20260620 09:21:16.123421 6762 ts_manager.cc:194] Registered new tserver with Master: 6334af0dbdfb4c4cb6afa3a3c5ae19ff (127.4.59.1:34671)
I20260620 09:21:16.124025 6762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:39587
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:21:16.144912 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:40441
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:16.225842 6958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:16.226053 6958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:16.226075 6958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:16.227319 6958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:16.227372 6958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:16.227399 6958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:21:16.228814 6958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:40441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:16.229041 6958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:16.229269 6958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:16.234288 6963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:16.234246 6966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:16.234246 6964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:16.234449 6958 server_base.cc:1034] running on GCE node
I20260620 09:21:16.234748 6958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:16.235026 6958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:16.236188 6958 hybrid_clock.cc:648] HybridClock initialized: now 1781947276236167 us; error 38 us; skew 500 ppm
Jun 20 09:21:16 dist-test-slave-414w krb5kdc[6717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947276, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:16.239032 6958 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:21:16.239514 6958 webserver.cc:460] Webserver started at http://127.4.59.2:36467/ using document root <none> and password file <none>
I20260620 09:21:16.239745 6958 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:16.239825 6958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:16.239984 6958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:16.241014 6958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "72502584a8a44fde93ba3a35e501608b"
format_stamp: "Formatted at 2026-06-20 09:21:16 on dist-test-slave-414w"
server_key: "8b41e3efd3862b91960405f17dea6a57"
server_key_iv: "ab07ee41f69fbf373d18cb1c0f370466"
server_key_version: "encryptionkey@0"
I20260620 09:21:16.241403 6958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "72502584a8a44fde93ba3a35e501608b"
format_stamp: "Formatted at 2026-06-20 09:21:16 on dist-test-slave-414w"
server_key: "8b41e3efd3862b91960405f17dea6a57"
server_key_iv: "ab07ee41f69fbf373d18cb1c0f370466"
server_key_version: "encryptionkey@0"
I20260620 09:21:16.242700 6958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:16.243439 6973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:16.243599 6958 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:16.243695 6958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "72502584a8a44fde93ba3a35e501608b"
format_stamp: "Formatted at 2026-06-20 09:21:16 on dist-test-slave-414w"
server_key: "8b41e3efd3862b91960405f17dea6a57"
server_key_iv: "ab07ee41f69fbf373d18cb1c0f370466"
server_key_version: "encryptionkey@0"
I20260620 09:21:16.243788 6958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:16.257933 6958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:16.259855 6958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:16.260020 6958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:16.260263 6958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:16.260571 6958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:16.260624 6958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:16.260659 6958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:16.260686 6958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:16.267445 6958 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:40435
I20260620 09:21:16.267491 7086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:40435 every 8 connection(s)
I20260620 09:21:16.267805 6958 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:21:16.269907 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 6958
I20260620 09:21:16.269982 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:21:16.270171 4332 external_mini_cluster.cc:1428] Setting key a16bc9c5f9ac01bbbc2e2fdb57c0407d
Jun 20 09:21:16 dist-test-slave-414w krb5kdc[6717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947276, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
I20260620 09:21:16.279043 6801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:16.268555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:54467 (local address 127.4.59.62:40441)
0620 09:21:16.268627 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:16.268629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:16.269051 (+ 422us) server_negotiation.cc:407] Connection header received
0620 09:21:16.269623 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:16.269624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:16.269635 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:16.269706 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:16.271111 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:16.271948 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:16.273002 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:16.273352 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:16.273439 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:16.275893 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:16.275894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:16.275894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:16.275907 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:16.277490 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:16.277933 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:16.277935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:16.277935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:16.277995 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:16.278281 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:16.278282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:16.278282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:16.278444 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:16.278534 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:16.278842 (+ 308us) server_negotiation.cc:299] Negotiation successful
0620 09:21:16.278926 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 09:21:16.279135 7089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:16.268541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40441 (local address 127.4.59.2:54467)
0620 09:21:16.268727 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:16.268823 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:16.269532 (+ 709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:16.269808 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:16.269812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:16.270639 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:16.270918 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:16.270924 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:16.272039 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:16.272040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:16.272919 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:16.272921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:16.273439 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:16.273440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:16.273531 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:16.274073 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:16.274082 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:16.275770 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:16.277594 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:16.277600 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:16.277603 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:16.277840 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:16.278063 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:16.278064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:16.278064 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:16.278192 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:16.278546 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:16.278552 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:16.278684 (+ 132us) client_negotiation.cc:769] Sending connection context
0620 09:21:16.278797 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 09:21:16.278927 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":102,"threads_started":1}
I20260620 09:21:16.279687 7087 heartbeater.cc:347] Connected to a master server at 127.4.59.62:40441
I20260620 09:21:16.279799 7087 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:16.280014 7087 heartbeater.cc:511] Master 127.4.59.62:40441 requested a full tablet report, sending...
I20260620 09:21:16.280642 6762 ts_manager.cc:194] Registered new tserver with Master: 72502584a8a44fde93ba3a35e501608b (127.4.59.2:40435)
I20260620 09:21:16.281339 6762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:54467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:21:16.296151 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:40441
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:16.377224 7092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:16.377481 7092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:16.377524 7092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:16.378705 7092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:16.378768 7092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:16.378813 7092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:21:16.380300 7092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:40441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:16.380553 7092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:16.380775 7092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:16.385445 7097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:16.385474 7098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:16.385617 7092 server_base.cc:1034] running on GCE node
W20260620 09:21:16.385500 7100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:16.385886 7092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:16.386133 7092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:16.387279 7092 hybrid_clock.cc:648] HybridClock initialized: now 1781947276387257 us; error 36 us; skew 500 ppm
Jun 20 09:21:16 dist-test-slave-414w krb5kdc[6717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947276, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:16.389750 7092 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:21:16.390173 7092 webserver.cc:460] Webserver started at http://127.4.59.3:43967/ using document root <none> and password file <none>
I20260620 09:21:16.390427 7092 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:16.390498 7092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:16.390633 7092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:16.391671 7092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "d759845a86ee443c8896941a3e74b301"
format_stamp: "Formatted at 2026-06-20 09:21:16 on dist-test-slave-414w"
server_key: "c0084035f17acdc70dd749eb30049219"
server_key_iv: "935baf7e242c958ad6d6288a122641c4"
server_key_version: "encryptionkey@0"
I20260620 09:21:16.392062 7092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "d759845a86ee443c8896941a3e74b301"
format_stamp: "Formatted at 2026-06-20 09:21:16 on dist-test-slave-414w"
server_key: "c0084035f17acdc70dd749eb30049219"
server_key_iv: "935baf7e242c958ad6d6288a122641c4"
server_key_version: "encryptionkey@0"
I20260620 09:21:16.393494 7092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 09:21:16.394302 7107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:16.394495 7092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:16.394593 7092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "d759845a86ee443c8896941a3e74b301"
format_stamp: "Formatted at 2026-06-20 09:21:16 on dist-test-slave-414w"
server_key: "c0084035f17acdc70dd749eb30049219"
server_key_iv: "935baf7e242c958ad6d6288a122641c4"
server_key_version: "encryptionkey@0"
I20260620 09:21:16.394680 7092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:16.417253 7092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:16.419271 7092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:16.419443 7092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:16.419687 7092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:16.420006 7092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:16.420060 7092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:16.420094 7092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:16.420122 7092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:16.425805 7092 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:42443
I20260620 09:21:16.425875 7220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:42443 every 8 connection(s)
I20260620 09:21:16.426200 7092 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:21:16.430826 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 7092
I20260620 09:21:16.430907 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:21:16.431082 4332 external_mini_cluster.cc:1428] Setting key ea226a1fdb50e7ed27fd63c11a2eb833
Jun 20 09:21:16 dist-test-slave-414w krb5kdc[6717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947276, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:16.435658 6801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:16.427183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:43245 (local address 127.4.59.62:40441)
0620 09:21:16.427240 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:16.427242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:16.427252 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:21:16.427717 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:16.427718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:16.427728 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:16.427794 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:16.428640 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:16.429190 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:16.430230 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:16.430431 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:16.430496 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:16.432772 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:16.432773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:16.432773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:16.432779 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:16.434145 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:16.434558 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:16.434559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:16.434559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:16.434613 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:16.435014 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:16.435016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:16.435017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:16.435182 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:16.435240 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:16.435528 (+ 288us) server_negotiation.cc:299] Negotiation successful
0620 09:21:16.435596 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 09:21:16.435933 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:16.426953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40441 (local address 127.4.59.3:43245)
0620 09:21:16.427104 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:16.427144 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:16.427613 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:16.427846 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:16.427849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:16.428381 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:16.428559 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:16.428563 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:16.429277 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:16.429278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:16.430140 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:16.430142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:16.430510 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:16.430510 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:16.430601 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:16.431114 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:16.431122 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:16.432671 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:16.434224 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:16.434229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:16.434231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:16.434471 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:16.434721 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:16.434722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:16.434723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:16.434854 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:16.435257 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:16.435263 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:16.435384 (+ 121us) client_negotiation.cc:769] Sending connection context
0620 09:21:16.435476 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 09:21:16.435589 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
I20260620 09:21:16.436479 7221 heartbeater.cc:347] Connected to a master server at 127.4.59.62:40441
I20260620 09:21:16.436591 7221 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:16.436780 7221 heartbeater.cc:511] Master 127.4.59.62:40441 requested a full tablet report, sending...
I20260620 09:21:16.437208 6762 ts_manager.cc:194] Registered new tserver with Master: d759845a86ee443c8896941a3e74b301 (127.4.59.3:42443)
I20260620 09:21:16.437597 6762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:43245
I20260620 09:21:16.442703 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 09:21:16.453832 6801 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:21:16.452885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54528 (local address 127.4.59.62:40441)
0620 09:21:16.452947 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:16.452950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:16.453052 (+ 102us) server_negotiation.cc:407] Connection header received
0620 09:21:16.453135 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:16.453136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:16.453187 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:16.453259 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:16.453759 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54528: BlockingRecv error: recv got EOF from 127.0.0.1:54528 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 09:21:16.454758 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 6824
I20260620 09:21:16.458971 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 6958
I20260620 09:21:16.463065 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 7092
I20260620 09:21:16.467217 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 6730
2026-06-20T09:21:16Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1833 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:21:16 dist-test-slave-414w krb5kdc[7233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:21:16 dist-test-slave-414w krb5kdc[7233](info): set up 2 sockets
Jun 20 09:21:16 dist-test-slave-414w krb5kdc[7233](info): commencing operation
krb5kdc: starting...
W20260620 09:21:17.509353 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:21:17 dist-test-slave-414w krb5kdc[7233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:21:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:21:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:21:17.626216 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:39721
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:39721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:17.707398 7246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:17.707597 7246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:17.707619 7246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:17.708774 7246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:17.708815 7246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:17.708830 7246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:17.708843 7246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:17.708855 7246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:21:17.710471 7246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:39721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:39721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:17.710721 7246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:17.710947 7246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:17.715830 7251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:17.715854 7254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:17.715847 7252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:17.715888 7246 server_base.cc:1034] running on GCE node
I20260620 09:21:17.716338 7246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:17.716626 7246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:17.717795 7246 hybrid_clock.cc:648] HybridClock initialized: now 1781947277717786 us; error 46 us; skew 500 ppm
Jun 20 09:21:17 dist-test-slave-414w krb5kdc[7233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947277, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:17.724447 7246 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:21:17.724917 7246 webserver.cc:460] Webserver started at http://127.4.59.62:38165/ using document root <none> and password file <none>
I20260620 09:21:17.725195 7246 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:17.725279 7246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:17.725438 7246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:17.726501 7246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "f5ff2cccda2141f880a09e51ac772f46"
format_stamp: "Formatted at 2026-06-20 09:21:17 on dist-test-slave-414w"
server_key: "4c3d1f7f3c5f2057702b677d0e52259b"
server_key_iv: "3ac5d349b8cfa8742cb62e1d76470777"
server_key_version: "encryptionkey@0"
I20260620 09:21:17.726871 7246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "f5ff2cccda2141f880a09e51ac772f46"
format_stamp: "Formatted at 2026-06-20 09:21:17 on dist-test-slave-414w"
server_key: "4c3d1f7f3c5f2057702b677d0e52259b"
server_key_iv: "3ac5d349b8cfa8742cb62e1d76470777"
server_key_version: "encryptionkey@0"
I20260620 09:21:17.728155 7246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:21:17.728855 7261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:17.729027 7246 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:21:17.729126 7246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "f5ff2cccda2141f880a09e51ac772f46"
format_stamp: "Formatted at 2026-06-20 09:21:17 on dist-test-slave-414w"
server_key: "4c3d1f7f3c5f2057702b677d0e52259b"
server_key_iv: "3ac5d349b8cfa8742cb62e1d76470777"
server_key_version: "encryptionkey@0"
I20260620 09:21:17.729216 7246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:17.743254 7246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:17.745220 7246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:17.745381 7246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:17.748929 7246 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:39721
I20260620 09:21:17.749007 7313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:39721 every 8 connection(s)
I20260620 09:21:17.749394 7246 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:21:17.750249 7314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:17.750927 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 7246
I20260620 09:21:17.751042 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:21:17.751230 4332 external_mini_cluster.cc:1428] Setting key 6617355516750a7d5a014d5724780fb1
I20260620 09:21:17.753002 7314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46: Bootstrap starting.
I20260620 09:21:17.753743 7314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:17.754048 7314 log.cc:824] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:17.754760 7314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46: No bootstrap required, opened a new log
I20260620 09:21:17.756968 7314 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ff2cccda2141f880a09e51ac772f46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39721 } }
I20260620 09:21:17.757100 7314 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:17.757122 7314 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ff2cccda2141f880a09e51ac772f46, State: Initialized, Role: FOLLOWER
I20260620 09:21:17.757297 7314 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ff2cccda2141f880a09e51ac772f46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39721 } }
I20260620 09:21:17.757376 7314 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:21:17.757400 7314 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:21:17.757469 7314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:17.758026 7314 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ff2cccda2141f880a09e51ac772f46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39721 } }
I20260620 09:21:17.758140 7314 leader_election.cc:302] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5ff2cccda2141f880a09e51ac772f46; no voters:
I20260620 09:21:17.758304 7314 leader_election.cc:288] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:21:17.758368 7319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:17.758531 7319 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [term 1 LEADER]: Becoming Leader. State: Replica: f5ff2cccda2141f880a09e51ac772f46, State: Running, Role: LEADER
Jun 20 09:21:17 dist-test-slave-414w krb5kdc[7233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:17.758551 7314 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:17.760017 7319 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ff2cccda2141f880a09e51ac772f46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39721 } }
I20260620 09:21:17.760624 7320 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5ff2cccda2141f880a09e51ac772f46. Latest consensus state: current_term: 1 leader_uuid: "f5ff2cccda2141f880a09e51ac772f46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ff2cccda2141f880a09e51ac772f46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39721 } } }
I20260620 09:21:17.760720 7321 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5ff2cccda2141f880a09e51ac772f46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ff2cccda2141f880a09e51ac772f46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39721 } } }
I20260620 09:21:17.760996 7320 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:17.761014 7321 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46 [sys.catalog]: This master's current role is: LEADER
W20260620 09:21:17.761328 7333 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:21:17.761397 7333 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:21:17.761478 7335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:21:17.761662 7335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:21:17.763029 7317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:17.751740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.4.59.62:39721)
0620 09:21:17.751923 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:17.751932 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:17.751968 (+ 36us) server_negotiation.cc:407] Connection header received
0620 09:21:17.752376 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:17.752382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:17.752428 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:17.752576 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:17.753322 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:17.754674 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:17.755645 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:17.756273 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:17.756408 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:17.759087 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:17.759091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:17.759091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:17.759109 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:17.761340 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:17.761770 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:17.761773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:17.761773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:17.761869 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:17.762106 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:17.762107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:17.762108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:17.762401 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:17.762560 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:17.762699 (+ 139us) server_negotiation.cc:299] Negotiation successful
0620 09:21:17.762838 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":77,"threads_started":1}
I20260620 09:21:17.763276 7335 catalog_manager.cc:1347] Generated new cluster ID: ca2a998fb2764313999bf1e95a2b54fe
I20260620 09:21:17.763321 7335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:21:17.765920 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:21:17.771421 7335 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:21:17.772114 7335 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:21:17.777792 7335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5ff2cccda2141f880a09e51ac772f46: Generated new TSK 0
I20260620 09:21:17.778002 7335 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:21:17.779945 7317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:17.776165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43170 (local address 127.4.59.62:39721)
0620 09:21:17.776210 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:17.776212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:17.776358 (+ 146us) server_negotiation.cc:407] Connection header received
0620 09:21:17.776438 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:17.776439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:17.776452 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:17.776510 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:17.776887 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:17.777526 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:17.778430 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:17.778669 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:17.778754 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:17.779489 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:17.779490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:17.779490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:21:17.779677 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:17.779692 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:21:17.779698 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:17.779761 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:17.779833 (+ 72us) server_negotiation.cc:299] Negotiation successful
0620 09:21:17.779858 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 09:21:17.782284 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 7246
2026-06-20T09:21:17Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1313 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:21:17 dist-test-slave-414w krb5kdc[7347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:21:17 dist-test-slave-414w krb5kdc[7347](info): set up 2 sockets
Jun 20 09:21:17 dist-test-slave-414w krb5kdc[7347](info): commencing operation
krb5kdc: starting...
W20260620 09:21:18.817415 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:21:18 dist-test-slave-414w krb5kdc[7347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:21:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:21:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:21:18.930034 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:36289
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:36289
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:19.009770 7360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:19.009963 7360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:19.009984 7360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:19.011124 7360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:19.011165 7360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:19.011179 7360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:19.011193 7360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:19.011241 7360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:21:19.012635 7360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:36289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:36289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:19.012825 7360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:19.012979 7360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:19.017817 7368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:19.017854 7366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:19.017948 7365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:19.017845 7360 server_base.cc:1034] running on GCE node
I20260620 09:21:19.018265 7360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:19.018553 7360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:19.019730 7360 hybrid_clock.cc:648] HybridClock initialized: now 1781947279019714 us; error 26 us; skew 500 ppm
Jun 20 09:21:19 dist-test-slave-414w krb5kdc[7347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947279, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:19.022207 7360 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:21:19.022647 7360 webserver.cc:460] Webserver started at http://127.4.59.62:34659/ using document root <none> and password file <none>
I20260620 09:21:19.022825 7360 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:19.022864 7360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:19.022946 7360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:19.023913 7360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "55259f44b6a5450592863db88fd9617e"
format_stamp: "Formatted at 2026-06-20 09:21:19 on dist-test-slave-414w"
server_key: "c3a010d459a1851dda1ce6ac035bdf19"
server_key_iv: "6277441dc1f226384abde7a1b76152d2"
server_key_version: "encryptionkey@0"
I20260620 09:21:19.024289 7360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "55259f44b6a5450592863db88fd9617e"
format_stamp: "Formatted at 2026-06-20 09:21:19 on dist-test-slave-414w"
server_key: "c3a010d459a1851dda1ce6ac035bdf19"
server_key_iv: "6277441dc1f226384abde7a1b76152d2"
server_key_version: "encryptionkey@0"
I20260620 09:21:19.025674 7360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:21:19.026464 7375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:19.026674 7360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:21:19.026759 7360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "55259f44b6a5450592863db88fd9617e"
format_stamp: "Formatted at 2026-06-20 09:21:19 on dist-test-slave-414w"
server_key: "c3a010d459a1851dda1ce6ac035bdf19"
server_key_iv: "6277441dc1f226384abde7a1b76152d2"
server_key_version: "encryptionkey@0"
I20260620 09:21:19.026815 7360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:19.042798 7360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:19.045032 7360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:19.045210 7360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:19.049060 7360 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:36289
I20260620 09:21:19.049129 7427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:36289 every 8 connection(s)
I20260620 09:21:19.049472 7360 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:21:19.050459 7428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:19.052578 7428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e: Bootstrap starting.
I20260620 09:21:19.053069 7428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:19.053378 7428 log.cc:824] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:19.053996 7428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e: No bootstrap required, opened a new log
I20260620 09:21:19.054634 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 7360
I20260620 09:21:19.054718 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:21:19.054878 4332 external_mini_cluster.cc:1428] Setting key e98a3afe738baf37f036cc862971f533
I20260620 09:21:19.055886 7428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55259f44b6a5450592863db88fd9617e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 36289 } }
I20260620 09:21:19.056046 7428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:19.056084 7428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55259f44b6a5450592863db88fd9617e, State: Initialized, Role: FOLLOWER
I20260620 09:21:19.056273 7428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55259f44b6a5450592863db88fd9617e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 36289 } }
I20260620 09:21:19.056368 7428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:21:19.056403 7428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:21:19.056520 7428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:19.057485 7428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55259f44b6a5450592863db88fd9617e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 36289 } }
I20260620 09:21:19.057569 7428 leader_election.cc:302] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55259f44b6a5450592863db88fd9617e; no voters:
I20260620 09:21:19.057778 7428 leader_election.cc:288] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:21:19.057801 7433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:19.057963 7433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [term 1 LEADER]: Becoming Leader. State: Replica: 55259f44b6a5450592863db88fd9617e, State: Running, Role: LEADER
I20260620 09:21:19.058027 7428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:19.058043 7433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55259f44b6a5450592863db88fd9617e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 36289 } }
I20260620 09:21:19.058329 7433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55259f44b6a5450592863db88fd9617e. Latest consensus state: current_term: 1 leader_uuid: "55259f44b6a5450592863db88fd9617e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55259f44b6a5450592863db88fd9617e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 36289 } } }
I20260620 09:21:19.058491 7433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:19.058344 7435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55259f44b6a5450592863db88fd9617e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55259f44b6a5450592863db88fd9617e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 36289 } } }
I20260620 09:21:19.058732 7435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:19.059290 7440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:21:19.059778 7440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:21:19.061833 7440 catalog_manager.cc:1347] Generated new cluster ID: b06cfd0194c1445197ef6c24a23c8054
I20260620 09:21:19.061888 7440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 09:21:19 dist-test-slave-414w krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:19.066284 7432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:19.055375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54008 (local address 127.4.59.62:36289)
0620 09:21:19.055566 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:19.055573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:19.055600 (+ 27us) server_negotiation.cc:407] Connection header received
0620 09:21:19.055868 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:19.055874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:19.055924 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:19.056028 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:19.056849 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:19.059127 (+ 2278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:19.060093 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:19.060496 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:19.060619 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:19.063108 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:19.063111 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:19.063112 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:19.063123 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:19.064767 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:19.065181 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:19.065185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:19.065185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:19.065280 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:19.065505 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:19.065506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:19.065506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:19.065726 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:19.065906 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:19.065968 (+ 62us) server_negotiation.cc:299] Negotiation successful
0620 09:21:19.066101 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260620 09:21:19.071170 7440 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:21:19.071856 7440 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:21:19.077800 7440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55259f44b6a5450592863db88fd9617e: Generated new TSK 0
I20260620 09:21:19.077960 7440 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:21:19.104022 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:36289
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:19.184005 7454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:19.184233 7454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:19.184273 7454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:19.185488 7454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:19.185551 7454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:19.185596 7454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:21:19.187018 7454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:36289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:19.187279 7454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:19.187496 7454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:19.192212 7460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:19.192240 7459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:19.192327 7454 server_base.cc:1034] running on GCE node
W20260620 09:21:19.192194 7462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:19.192634 7454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:19.192873 7454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:19.194010 7454 hybrid_clock.cc:648] HybridClock initialized: now 1781947279193997 us; error 30 us; skew 500 ppm
Jun 20 09:21:19 dist-test-slave-414w krb5kdc[7347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947279, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:19.196379 7454 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:21:19.196774 7454 webserver.cc:460] Webserver started at http://127.4.59.1:44979/ using document root <none> and password file <none>
I20260620 09:21:19.196995 7454 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:19.197063 7454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:19.197223 7454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:19.198199 7454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "80c906f2674d4a109037bb48a32f1b9f"
format_stamp: "Formatted at 2026-06-20 09:21:19 on dist-test-slave-414w"
server_key: "d70ae8b8c1df5cbc8bd1033a1b8fbc66"
server_key_iv: "8ec831338a02691588a5983e8241c19f"
server_key_version: "encryptionkey@0"
I20260620 09:21:19.198551 7454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "80c906f2674d4a109037bb48a32f1b9f"
format_stamp: "Formatted at 2026-06-20 09:21:19 on dist-test-slave-414w"
server_key: "d70ae8b8c1df5cbc8bd1033a1b8fbc66"
server_key_iv: "8ec831338a02691588a5983e8241c19f"
server_key_version: "encryptionkey@0"
I20260620 09:21:19.199828 7454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 09:21:19.200556 7469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:19.200717 7454 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:21:19.200824 7454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "80c906f2674d4a109037bb48a32f1b9f"
format_stamp: "Formatted at 2026-06-20 09:21:19 on dist-test-slave-414w"
server_key: "d70ae8b8c1df5cbc8bd1033a1b8fbc66"
server_key_iv: "8ec831338a02691588a5983e8241c19f"
server_key_version: "encryptionkey@0"
I20260620 09:21:19.200894 7454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:19.219296 7454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:19.221007 7454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:19.221109 7454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:19.221319 7454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:19.221547 7454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:19.221575 7454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:19.221596 7454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:19.221608 7454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:19.226972 7454 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:44579
I20260620 09:21:19.227008 7582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:44579 every 8 connection(s)
I20260620 09:21:19.227319 7454 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:21:19.228488 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 7454
I20260620 09:21:19.228578 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:21:19.228761 4332 external_mini_cluster.cc:1428] Setting key fd20c292ebf57696a1fb291031a5964c
Jun 20 09:21:19 dist-test-slave-414w krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947279, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
I20260620 09:21:19.236690 7432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:19.228017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:59887 (local address 127.4.59.62:36289)
0620 09:21:19.228077 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:19.228079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:19.228270 (+ 191us) server_negotiation.cc:407] Connection header received
0620 09:21:19.228822 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:19.228824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:19.228837 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:19.228882 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:19.229920 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:19.230592 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:19.231558 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:19.231784 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:19.231858 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:19.233983 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:19.233984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:19.233985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:19.233993 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:19.235305 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:19.235720 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:19.235721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:19.235721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:19.235785 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:19.236040 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:19.236040 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:19.236041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:19.236147 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:19.236221 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:19.236514 (+ 293us) server_negotiation.cc:299] Negotiation successful
0620 09:21:19.236601 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 09:21:19.236841 7585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:19.227990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:36289 (local address 127.4.59.1:59887)
0620 09:21:19.228162 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:19.228193 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:19.228714 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:19.228957 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:19.228963 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:19.229637 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:19.229815 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:19.229819 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:19.230682 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:19.230683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:19.231456 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:19.231458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:19.231864 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:19.231865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:19.231994 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:19.232494 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:19.232505 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:19.233888 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:19.235404 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:19.235411 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:19.235413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:19.235629 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:19.235850 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:19.235851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:19.235852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:19.235963 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:19.236212 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:19.236218 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:19.236375 (+ 157us) client_negotiation.cc:769] Sending connection context
0620 09:21:19.236476 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 09:21:19.236623 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260620 09:21:19.237376 7583 heartbeater.cc:347] Connected to a master server at 127.4.59.62:36289
I20260620 09:21:19.237454 7583 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:19.237630 7583 heartbeater.cc:511] Master 127.4.59.62:36289 requested a full tablet report, sending...
I20260620 09:21:19.238078 7392 ts_manager.cc:194] Registered new tserver with Master: 80c906f2674d4a109037bb48a32f1b9f (127.4.59.1:44579)
I20260620 09:21:19.238669 7392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:59887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:21:19.252125 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:36289
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:19.332484 7588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:19.332679 7588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:19.332701 7588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:19.333791 7588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:19.333822 7588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:19.333847 7588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:21:19.335155 7588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:36289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:19.335331 7588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:19.335489 7588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:19.340427 7594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:19.340437 7596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:19.340584 7588 server_base.cc:1034] running on GCE node
W20260620 09:21:19.340462 7593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:19.340893 7588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:19.341087 7588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:19.342242 7588 hybrid_clock.cc:648] HybridClock initialized: now 1781947279342228 us; error 29 us; skew 500 ppm
Jun 20 09:21:19 dist-test-slave-414w krb5kdc[7347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947279, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:19.344815 7588 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:21:19.345321 7588 webserver.cc:460] Webserver started at http://127.4.59.2:35901/ using document root <none> and password file <none>
I20260620 09:21:19.345542 7588 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:19.345585 7588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:19.345738 7588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:19.346745 7588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "37e5b22012854d9b9493ab8de049b963"
format_stamp: "Formatted at 2026-06-20 09:21:19 on dist-test-slave-414w"
server_key: "d79768da450d9827abbde158c7b5276a"
server_key_iv: "139ac1bc72ebb0245f804856801b3891"
server_key_version: "encryptionkey@0"
I20260620 09:21:19.347095 7588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "37e5b22012854d9b9493ab8de049b963"
format_stamp: "Formatted at 2026-06-20 09:21:19 on dist-test-slave-414w"
server_key: "d79768da450d9827abbde158c7b5276a"
server_key_iv: "139ac1bc72ebb0245f804856801b3891"
server_key_version: "encryptionkey@0"
I20260620 09:21:19.348305 7588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:21:19.348946 7603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:19.349179 7588 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:21:19.349273 7588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "37e5b22012854d9b9493ab8de049b963"
format_stamp: "Formatted at 2026-06-20 09:21:19 on dist-test-slave-414w"
server_key: "d79768da450d9827abbde158c7b5276a"
server_key_iv: "139ac1bc72ebb0245f804856801b3891"
server_key_version: "encryptionkey@0"
I20260620 09:21:19.349344 7588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:19.367916 7588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:19.369928 7588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:19.370070 7588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:19.370306 7588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:19.370608 7588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:19.370641 7588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:19.370682 7588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:19.370709 7588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:19.376698 7588 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:45457
I20260620 09:21:19.376780 7716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:45457 every 8 connection(s)
I20260620 09:21:19.377108 7588 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
Jun 20 09:21:19 dist-test-slave-414w krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947279, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:19.386392 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 7588
I20260620 09:21:19.386512 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:21:19.386739 4332 external_mini_cluster.cc:1428] Setting key fdbd42f06f27b20d8197cb72ed9f0d40
I20260620 09:21:19.387172 7432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:19.378195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:48849 (local address 127.4.59.62:36289)
0620 09:21:19.378285 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:19.378288 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:19.378390 (+ 102us) server_negotiation.cc:407] Connection header received
0620 09:21:19.378883 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:19.378891 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:19.378902 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:19.378955 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:19.379912 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:19.380411 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:19.381457 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:19.381655 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:19.381722 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:19.383900 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:19.383901 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:19.383902 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:19.383910 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:19.385599 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:19.386033 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:19.386034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:19.386034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:19.386098 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:19.386368 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:19.386369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:19.386370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:19.386508 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:19.386576 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:19.386957 (+ 381us) server_negotiation.cc:299] Negotiation successful
0620 09:21:19.387049 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 09:21:19.387347 7719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:19.378088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:36289 (local address 127.4.59.2:48849)
0620 09:21:19.378263 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:19.378312 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:19.378790 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:19.379015 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:19.379018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:19.379615 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:19.379805 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:19.379809 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:19.380497 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:19.380497 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:19.381362 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:19.381364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:19.381735 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:19.381735 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:19.381827 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:19.382400 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:19.382408 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:19.383805 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:19.385720 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:19.385727 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:19.385729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:19.385954 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:19.386190 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:19.386191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:19.386191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:19.386298 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:19.386611 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:19.386618 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:19.386790 (+ 172us) client_negotiation.cc:769] Sending connection context
0620 09:21:19.386921 (+ 131us) client_negotiation.cc:240] Negotiation successful
0620 09:21:19.387112 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260620 09:21:19.387881 7717 heartbeater.cc:347] Connected to a master server at 127.4.59.62:36289
I20260620 09:21:19.387979 7717 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:19.388177 7717 heartbeater.cc:511] Master 127.4.59.62:36289 requested a full tablet report, sending...
I20260620 09:21:19.388538 7392 ts_manager.cc:194] Registered new tserver with Master: 37e5b22012854d9b9493ab8de049b963 (127.4.59.2:45457)
I20260620 09:21:19.388968 7392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:48849
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:21:19.412249 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:36289
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:19.496031 7722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:19.496227 7722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:19.496249 7722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:19.497453 7722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:19.497493 7722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:19.497519 7722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:21:19.498919 7722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:36289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:19.499158 7722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:19.499377 7722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:19.504215 7727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:19.504245 7730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:19.504232 7728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:19.504447 7722 server_base.cc:1034] running on GCE node
I20260620 09:21:19.504637 7722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:19.504820 7722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:19.505954 7722 hybrid_clock.cc:648] HybridClock initialized: now 1781947279505940 us; error 29 us; skew 500 ppm
Jun 20 09:21:19 dist-test-slave-414w krb5kdc[7347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947279, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:19.508460 7722 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:21:19.508857 7722 webserver.cc:460] Webserver started at http://127.4.59.3:44279/ using document root <none> and password file <none>
I20260620 09:21:19.509071 7722 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:19.509136 7722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:19.509310 7722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:19.510305 7722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "836991120255439ba7762132a84be062"
format_stamp: "Formatted at 2026-06-20 09:21:19 on dist-test-slave-414w"
server_key: "0ae73a0f4cdbe656251f170581d123e4"
server_key_iv: "76c1e8642b590aecdeff7de02631d3ee"
server_key_version: "encryptionkey@0"
I20260620 09:21:19.510658 7722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "836991120255439ba7762132a84be062"
format_stamp: "Formatted at 2026-06-20 09:21:19 on dist-test-slave-414w"
server_key: "0ae73a0f4cdbe656251f170581d123e4"
server_key_iv: "76c1e8642b590aecdeff7de02631d3ee"
server_key_version: "encryptionkey@0"
I20260620 09:21:19.511924 7722 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:21:19.512590 7737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:19.512755 7722 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:19.512849 7722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "836991120255439ba7762132a84be062"
format_stamp: "Formatted at 2026-06-20 09:21:19 on dist-test-slave-414w"
server_key: "0ae73a0f4cdbe656251f170581d123e4"
server_key_iv: "76c1e8642b590aecdeff7de02631d3ee"
server_key_version: "encryptionkey@0"
I20260620 09:21:19.512928 7722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:19.526726 7722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:19.528573 7722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:19.528723 7722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:19.528949 7722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:19.529315 7722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:19.529366 7722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:19.529400 7722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:19.529428 7722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:19.535003 7722 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:46409
I20260620 09:21:19.535048 7850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:46409 every 8 connection(s)
I20260620 09:21:19.535331 7722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:21:19.537410 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 7722
I20260620 09:21:19.537506 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:21:19.537694 4332 external_mini_cluster.cc:1428] Setting key 20cd102566f1cc7c0f353d2fabfb09ce
Jun 20 09:21:19 dist-test-slave-414w krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947279, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:19.544346 7432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:19.535952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:44343 (local address 127.4.59.62:36289)
0620 09:21:19.535992 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:19.535994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:19.536203 (+ 209us) server_negotiation.cc:407] Connection header received
0620 09:21:19.536652 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:19.536653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:19.536667 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:19.536751 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:19.537833 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:19.538558 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:19.539498 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:19.539674 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:19.539736 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:19.542023 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:19.542024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:19.542025 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:19.542031 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:19.543211 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:19.543555 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:19.543555 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:19.543556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:19.543605 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:19.543827 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:19.543828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:19.543829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:19.543935 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:19.543993 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:19.544224 (+ 231us) server_negotiation.cc:299] Negotiation successful
0620 09:21:19.544282 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:21:19.544489 7853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:19.535950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:36289 (local address 127.4.59.3:44343)
0620 09:21:19.536087 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:19.536118 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:19.536559 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:19.536799 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:19.536802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:19.537472 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:19.537744 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:19.537751 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:19.538642 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:19.538642 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:19.539424 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:19.539426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:19.539783 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:19.539784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:19.539890 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:19.540330 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:19.540337 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:19.541945 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:19.543291 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:19.543296 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:19.543297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:19.543494 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:19.543661 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:19.543662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:19.543662 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:19.543748 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:19.543997 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:19.544001 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:19.544102 (+ 101us) client_negotiation.cc:769] Sending connection context
0620 09:21:19.544190 (+ 88us) client_negotiation.cc:240] Negotiation successful
0620 09:21:19.544298 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":81,"threads_started":1}
I20260620 09:21:19.544952 7851 heartbeater.cc:347] Connected to a master server at 127.4.59.62:36289
I20260620 09:21:19.545054 7851 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:19.545286 7851 heartbeater.cc:511] Master 127.4.59.62:36289 requested a full tablet report, sending...
I20260620 09:21:19.545676 7392 ts_manager.cc:194] Registered new tserver with Master: 836991120255439ba7762132a84be062 (127.4.59.3:46409)
I20260620 09:21:19.546077 7392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:44343
I20260620 09:21:19.549194 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 09:21:19 dist-test-slave-414w krb5kdc[7347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947279, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 20 09:21:19 dist-test-slave-414w krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947279, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:19.572305 7432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:19.565580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38022 (local address 127.4.59.62:36289)
0620 09:21:19.565655 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:19.565658 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:19.565769 (+ 111us) server_negotiation.cc:407] Connection header received
0620 09:21:19.565816 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:19.565817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:19.565826 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:19.565875 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:19.566613 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:19.567090 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:19.567774 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:19.567951 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:19.568004 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:19.570198 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:19.570200 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:19.570200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:19.570206 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:19.571396 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:19.571700 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:19.571701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:19.571701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:19.571749 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:19.571905 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:19.571906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:19.571906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:19.572035 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:19.572105 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:19.572187 (+ 82us) server_negotiation.cc:299] Negotiation successful
0620 09:21:19.572241 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260620 09:21:19.572676 7392 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38022
I20260620 09:21:19.573187 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 7454
I20260620 09:21:19.577802 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 7588
I20260620 09:21:19.581981 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 7722
I20260620 09:21:19.586229 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 7360
2026-06-20T09:21:19Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1805 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:21:20 dist-test-slave-414w krb5kdc[7863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:21:20 dist-test-slave-414w krb5kdc[7863](info): set up 2 sockets
Jun 20 09:21:20 dist-test-slave-414w krb5kdc[7863](info): commencing operation
krb5kdc: starting...
W20260620 09:21:21.969691 4332 mini_kdc.cc:121] Time spent starting KDC: real 2.363s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260620 09:21:22.127766 4332 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.158s user 0.001s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260620 09:21:22.442178 4332 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.314s user 0.000s sys 0.001s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260620 09:21:22.640384 4332 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.198s user 0.000s sys 0.001s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/test-user.keytab.
W20260620 09:21:23.708806 4332 mini_kdc.cc:302] Time spent creating keytab for test-user: real 1.068s user 0.000s sys 0.002s
Jun 20 09:21:23 dist-test-slave-414w krb5kdc[7863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:21:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:21:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:21:23.757624 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:36677
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:36677
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:23.842397 7876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:23.842648 7876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:23.842687 7876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:23.843870 7876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:23.843941 7876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:23.843966 7876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:23.843994 7876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:23.844026 7876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:21:23.845554 7876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:36677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:36677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:23.845836 7876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:23.846062 7876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:23.850904 7884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:23.850931 7882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:23.850932 7876 server_base.cc:1034] running on GCE node
W20260620 09:21:23.850904 7881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:23.851358 7876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:23.851639 7876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:23.852800 7876 hybrid_clock.cc:648] HybridClock initialized: now 1781947283852771 us; error 48 us; skew 500 ppm
Jun 20 09:21:23 dist-test-slave-414w krb5kdc[7863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947283, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:23.859722 7876 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:21:23.860152 7876 webserver.cc:460] Webserver started at http://127.4.59.62:43027/ using document root <none> and password file <none>
I20260620 09:21:23.860392 7876 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:23.860461 7876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:23.860615 7876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:23.861737 7876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "6781901e1908412ea3683f5bc4996cf1"
format_stamp: "Formatted at 2026-06-20 09:21:23 on dist-test-slave-414w"
server_key: "a63314a531cf50a995638e438a629698"
server_key_iv: "8272a650a376dce75e3cfc1b2798cbdb"
server_key_version: "encryptionkey@0"
I20260620 09:21:23.862152 7876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "6781901e1908412ea3683f5bc4996cf1"
format_stamp: "Formatted at 2026-06-20 09:21:23 on dist-test-slave-414w"
server_key: "a63314a531cf50a995638e438a629698"
server_key_iv: "8272a650a376dce75e3cfc1b2798cbdb"
server_key_version: "encryptionkey@0"
I20260620 09:21:23.863489 7876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:21:23.864231 7891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:23.864446 7876 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:23.864552 7876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "6781901e1908412ea3683f5bc4996cf1"
format_stamp: "Formatted at 2026-06-20 09:21:23 on dist-test-slave-414w"
server_key: "a63314a531cf50a995638e438a629698"
server_key_iv: "8272a650a376dce75e3cfc1b2798cbdb"
server_key_version: "encryptionkey@0"
I20260620 09:21:23.864632 7876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:23.875207 7876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:23.877447 7876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:23.877604 7876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:23.881326 7876 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:36677
I20260620 09:21:23.881366 7943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:36677 every 8 connection(s)
I20260620 09:21:23.881868 7876 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:21:23.882387 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 7876
I20260620 09:21:23.882490 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:21:23.882525 7944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:23.882684 4332 external_mini_cluster.cc:1428] Setting key 8c193e8f1be57a83bf49a469a048bcb2
I20260620 09:21:23.885641 7944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1: Bootstrap starting.
I20260620 09:21:23.886359 7944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:23.886647 7944 log.cc:824] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:23.887326 7944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1: No bootstrap required, opened a new log
Jun 20 09:21:23 dist-test-slave-414w krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:23.889810 7944 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6781901e1908412ea3683f5bc4996cf1" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 36677 } }
I20260620 09:21:23.889978 7944 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:23.890023 7944 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6781901e1908412ea3683f5bc4996cf1, State: Initialized, Role: FOLLOWER
I20260620 09:21:23.890156 7944 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6781901e1908412ea3683f5bc4996cf1" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 36677 } }
I20260620 09:21:23.890242 7944 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:21:23.890285 7944 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:21:23.890348 7944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:23.891009 7944 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6781901e1908412ea3683f5bc4996cf1" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 36677 } }
I20260620 09:21:23.891124 7944 leader_election.cc:302] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6781901e1908412ea3683f5bc4996cf1; no voters:
I20260620 09:21:23.891276 7944 leader_election.cc:288] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:21:23.891340 7949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:23.891532 7944 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:23.891539 7949 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [term 1 LEADER]: Becoming Leader. State: Replica: 6781901e1908412ea3683f5bc4996cf1, State: Running, Role: LEADER
I20260620 09:21:23.891745 7949 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6781901e1908412ea3683f5bc4996cf1" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 36677 } }
I20260620 09:21:23.892179 7950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6781901e1908412ea3683f5bc4996cf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6781901e1908412ea3683f5bc4996cf1" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 36677 } } }
I20260620 09:21:23.892171 7949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6781901e1908412ea3683f5bc4996cf1. Latest consensus state: current_term: 1 leader_uuid: "6781901e1908412ea3683f5bc4996cf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6781901e1908412ea3683f5bc4996cf1" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 36677 } } }
I20260620 09:21:23.892277 7950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:23.892284 7949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:23.892467 7958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:21:23.892985 7958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:21:23.894097 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:23.883201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46500 (local address 127.4.59.62:36677)
0620 09:21:23.883387 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:23.883394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:23.883418 (+ 24us) server_negotiation.cc:407] Connection header received
0620 09:21:23.883935 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:23.883941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:23.883986 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:23.884094 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:23.884877 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:23.885848 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:23.886911 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:23.887376 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:23.887501 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:23.890261 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:23.890265 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:23.890266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:23.890278 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:23.892411 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:23.892855 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:23.892858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:23.892858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:23.892969 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:23.893227 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:23.893228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:23.893228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:23.893499 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:23.893609 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:23.893773 (+ 164us) server_negotiation.cc:299] Negotiation successful
0620 09:21:23.893870 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260620 09:21:23.894973 7958 catalog_manager.cc:1347] Generated new cluster ID: a1263fd1c11542ae8ee9caf8ec5803cd
I20260620 09:21:23.895015 7958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:21:23.900250 7958 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:21:23.900717 7958 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:21:23.905836 7958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6781901e1908412ea3683f5bc4996cf1: Generated new TSK 0
I20260620 09:21:23.906004 7958 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:21:23.935046 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:36677
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:24.014014 7970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:24.014204 7970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:24.014226 7970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:24.015369 7970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:24.015409 7970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:24.015435 7970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:21:24.016906 7970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:36677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:24.017139 7970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:24.017387 7970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:24.022169 7976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:24.022199 7975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:24.022403 7978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:24.022449 7970 server_base.cc:1034] running on GCE node
I20260620 09:21:24.022650 7970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:24.022891 7970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:24.024031 7970 hybrid_clock.cc:648] HybridClock initialized: now 1781947284024009 us; error 37 us; skew 500 ppm
Jun 20 09:21:24 dist-test-slave-414w krb5kdc[7863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947284, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:24.026464 7970 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:21:24.026923 7970 webserver.cc:460] Webserver started at http://127.4.59.1:38709/ using document root <none> and password file <none>
I20260620 09:21:24.027141 7970 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:24.027207 7970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:24.027362 7970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:24.028367 7970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "253672c2f18a4ca18cc1158040823b5e"
format_stamp: "Formatted at 2026-06-20 09:21:24 on dist-test-slave-414w"
server_key: "e7e1a695b9483130fc06edb731f6fe56"
server_key_iv: "5f21852939b2d989a0f9db4328495e12"
server_key_version: "encryptionkey@0"
I20260620 09:21:24.028730 7970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "253672c2f18a4ca18cc1158040823b5e"
format_stamp: "Formatted at 2026-06-20 09:21:24 on dist-test-slave-414w"
server_key: "e7e1a695b9483130fc06edb731f6fe56"
server_key_iv: "5f21852939b2d989a0f9db4328495e12"
server_key_version: "encryptionkey@0"
I20260620 09:21:24.030085 7970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:21:24.030795 7985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:24.031057 7970 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:21:24.031164 7970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "253672c2f18a4ca18cc1158040823b5e"
format_stamp: "Formatted at 2026-06-20 09:21:24 on dist-test-slave-414w"
server_key: "e7e1a695b9483130fc06edb731f6fe56"
server_key_iv: "5f21852939b2d989a0f9db4328495e12"
server_key_version: "encryptionkey@0"
I20260620 09:21:24.031261 7970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:24.068296 7970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:24.070442 7970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:24.070604 7970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:24.070847 7970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:24.071158 7970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:24.071219 7970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:24.071261 7970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:24.071293 7970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:24.076897 7970 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:37071
I20260620 09:21:24.076946 8098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:37071 every 8 connection(s)
I20260620 09:21:24.077314 7970 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:21:24.079545 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 7970
I20260620 09:21:24.079643 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:21:24.079854 4332 external_mini_cluster.cc:1428] Setting key cdcb8cbf93621b1ad62cc79d1bdcd47c
Jun 20 09:21:24 dist-test-slave-414w krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947284, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
I20260620 09:21:24.087459 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:24.078061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:56597 (local address 127.4.59.62:36677)
0620 09:21:24.078125 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:24.078128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:24.078357 (+ 229us) server_negotiation.cc:407] Connection header received
0620 09:21:24.079272 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:24.079273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:24.079283 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:24.079381 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:24.080170 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:24.080897 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:24.082005 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:24.082348 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:24.082428 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:24.084540 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:24.084541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:24.084542 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:24.084549 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:24.086047 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:24.086479 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:24.086480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:24.086481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:24.086543 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:24.086795 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:24.086796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:24.086796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:24.086922 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:24.087004 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:24.087273 (+ 269us) server_negotiation.cc:299] Negotiation successful
0620 09:21:24.087365 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 09:21:24.087682 8101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:24.078039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:36677 (local address 127.4.59.1:56597)
0620 09:21:24.078211 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:24.078248 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:24.079160 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:24.079454 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:24.079458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:24.079804 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:24.080082 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:24.080089 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:24.080988 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:24.080989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:24.081920 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:24.081923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:24.082452 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:24.082453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:24.082557 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:24.083104 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:24.083113 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:24.084441 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:24.086153 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:24.086160 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:24.086162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:24.086383 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:24.086612 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:24.086613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:24.086613 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:24.086725 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:24.086993 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:24.087000 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:24.087140 (+ 140us) client_negotiation.cc:769] Sending connection context
0620 09:21:24.087251 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 09:21:24.087403 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260620 09:21:24.088212 8099 heartbeater.cc:347] Connected to a master server at 127.4.59.62:36677
I20260620 09:21:24.088313 8099 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:24.088511 8099 heartbeater.cc:511] Master 127.4.59.62:36677 requested a full tablet report, sending...
I20260620 09:21:24.089052 7908 ts_manager.cc:194] Registered new tserver with Master: 253672c2f18a4ca18cc1158040823b5e (127.4.59.1:37071)
I20260620 09:21:24.089725 7908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:56597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:21:24.103878 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:36677
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:24.182101 8104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:24.182283 8104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:24.182314 8104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:24.183380 8104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:24.183413 8104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:24.183439 8104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:21:24.184831 8104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:36677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:24.185005 8104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:24.185190 8104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 09:21:24.189906 8104 server_base.cc:1034] running on GCE node
W20260620 09:21:24.189880 8110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:24.189880 8112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:24.189869 8109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:24.190279 8104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:24.190524 8104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:24.191694 8104 hybrid_clock.cc:648] HybridClock initialized: now 1781947284191680 us; error 43 us; skew 500 ppm
Jun 20 09:21:24 dist-test-slave-414w krb5kdc[7863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947284, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:24.194258 8104 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:21:24.194725 8104 webserver.cc:460] Webserver started at http://127.4.59.2:34379/ using document root <none> and password file <none>
I20260620 09:21:24.194977 8104 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:24.195046 8104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:24.195196 8104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:24.196203 8104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "77451b7fc92946ee9b8d15af79fa0783"
format_stamp: "Formatted at 2026-06-20 09:21:24 on dist-test-slave-414w"
server_key: "303f1a55fd18776d7875bfc4e7aeccf9"
server_key_iv: "1b5dc57641cbfb39b8a821e5254a20bb"
server_key_version: "encryptionkey@0"
I20260620 09:21:24.196548 8104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "77451b7fc92946ee9b8d15af79fa0783"
format_stamp: "Formatted at 2026-06-20 09:21:24 on dist-test-slave-414w"
server_key: "303f1a55fd18776d7875bfc4e7aeccf9"
server_key_iv: "1b5dc57641cbfb39b8a821e5254a20bb"
server_key_version: "encryptionkey@0"
I20260620 09:21:24.197911 8104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:21:24.198588 8119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:24.198747 8104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:24.198843 8104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "77451b7fc92946ee9b8d15af79fa0783"
format_stamp: "Formatted at 2026-06-20 09:21:24 on dist-test-slave-414w"
server_key: "303f1a55fd18776d7875bfc4e7aeccf9"
server_key_iv: "1b5dc57641cbfb39b8a821e5254a20bb"
server_key_version: "encryptionkey@0"
I20260620 09:21:24.198923 8104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:24.214499 8104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:24.216259 8104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:24.216416 8104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:24.216624 8104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:24.216917 8104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:24.216972 8104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:24.217011 8104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:24.217039 8104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:24.222968 8104 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:41763
I20260620 09:21:24.223009 8232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:41763 every 8 connection(s)
I20260620 09:21:24.223308 8104 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:21:24.228245 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 8104
I20260620 09:21:24.228350 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:21:24.228540 4332 external_mini_cluster.cc:1428] Setting key 1a15307fd7325d47525f95eecd84e6d3
Jun 20 09:21:24 dist-test-slave-414w krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947284, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:24.232998 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:24.223974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:53763 (local address 127.4.59.62:36677)
0620 09:21:24.224019 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:24.224023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:24.224195 (+ 172us) server_negotiation.cc:407] Connection header received
0620 09:21:24.225077 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:24.225078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:24.225087 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:24.225205 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:24.225747 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:24.226238 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:24.227183 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:24.227364 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:24.227412 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:24.229913 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:24.229914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:24.229914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:24.229921 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:24.231644 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:24.232020 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:24.232021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:24.232021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:24.232085 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:24.232305 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:24.232306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:24.232306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:24.232453 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:24.232525 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:24.232759 (+ 234us) server_negotiation.cc:299] Negotiation successful
0620 09:21:24.232853 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 09:21:24.233053 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:24.223932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:36677 (local address 127.4.59.2:53763)
0620 09:21:24.224084 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:24.224123 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:24.224988 (+ 865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:24.225258 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:24.225261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:24.225493 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:24.225673 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:24.225677 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:24.226312 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:24.226312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:24.227094 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:24.227096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:24.227431 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:24.227431 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:24.227531 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:24.227998 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:24.228006 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:24.229806 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:24.231727 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:24.231732 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:24.231734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:24.231945 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:24.232153 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:24.232154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:24.232154 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:24.232241 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:24.232518 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:24.232521 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:24.232625 (+ 104us) client_negotiation.cc:769] Sending connection context
0620 09:21:24.232703 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 09:21:24.232865 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":91,"threads_started":1}
I20260620 09:21:24.233594 8233 heartbeater.cc:347] Connected to a master server at 127.4.59.62:36677
I20260620 09:21:24.233704 8233 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:24.233924 8233 heartbeater.cc:511] Master 127.4.59.62:36677 requested a full tablet report, sending...
I20260620 09:21:24.234277 7908 ts_manager.cc:194] Registered new tserver with Master: 77451b7fc92946ee9b8d15af79fa0783 (127.4.59.2:41763)
I20260620 09:21:24.234711 7908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:53763
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:21:24.253759 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:36677
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:24.333316 8238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:24.333547 8238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:24.333585 8238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:24.334774 8238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:24.334847 8238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:24.334890 8238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:21:24.336395 8238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:36677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:24.336660 8238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:24.336889 8238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:24.341784 8244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:24.341785 8243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:24.341995 8238 server_base.cc:1034] running on GCE node
W20260620 09:21:24.341861 8246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:24.342308 8238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:24.342548 8238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:24.343730 8238 hybrid_clock.cc:648] HybridClock initialized: now 1781947284343715 us; error 45 us; skew 500 ppm
Jun 20 09:21:24 dist-test-slave-414w krb5kdc[7863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947284, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:24.346277 8238 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:21:24.346657 8238 webserver.cc:460] Webserver started at http://127.4.59.3:38835/ using document root <none> and password file <none>
I20260620 09:21:24.346848 8238 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:24.346887 8238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:24.347090 8238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:24.348067 8238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "762c777210ef4312b983492b8791099e"
format_stamp: "Formatted at 2026-06-20 09:21:24 on dist-test-slave-414w"
server_key: "4806804741663b29f28a88c619ffd38f"
server_key_iv: "2fafdea57ed940212ed084d73bd776e3"
server_key_version: "encryptionkey@0"
I20260620 09:21:24.348353 8238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "762c777210ef4312b983492b8791099e"
format_stamp: "Formatted at 2026-06-20 09:21:24 on dist-test-slave-414w"
server_key: "4806804741663b29f28a88c619ffd38f"
server_key_iv: "2fafdea57ed940212ed084d73bd776e3"
server_key_version: "encryptionkey@0"
I20260620 09:21:24.349660 8238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:21:24.350346 8253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:24.350517 8238 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:24.350592 8238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "762c777210ef4312b983492b8791099e"
format_stamp: "Formatted at 2026-06-20 09:21:24 on dist-test-slave-414w"
server_key: "4806804741663b29f28a88c619ffd38f"
server_key_iv: "2fafdea57ed940212ed084d73bd776e3"
server_key_version: "encryptionkey@0"
I20260620 09:21:24.350646 8238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:24.360543 8238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:24.362385 8238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:24.362485 8238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:24.362641 8238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:24.362869 8238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:24.362896 8238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:24.362917 8238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:24.362929 8238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:24.368324 8238 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:42299
I20260620 09:21:24.368371 8366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:42299 every 8 connection(s)
I20260620 09:21:24.368667 8238 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
Jun 20 09:21:24 dist-test-slave-414w krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947284, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:24.377357 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:24.369390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:37021 (local address 127.4.59.62:36677)
0620 09:21:24.369440 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:24.369443 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:24.369660 (+ 217us) server_negotiation.cc:407] Connection header received
0620 09:21:24.370670 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:24.370671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:24.370685 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:24.370743 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:24.371307 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:24.371788 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:24.372702 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:24.372872 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:24.372933 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:24.375028 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:24.375029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:24.375030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:24.375036 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:24.376190 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:24.376524 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:24.376525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:24.376525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:24.376574 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:24.376775 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:24.376775 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:24.376776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:24.376866 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:24.376915 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:24.377239 (+ 324us) server_negotiation.cc:299] Negotiation successful
0620 09:21:24.377297 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:21:24.377482 8369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:24.369389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:36677 (local address 127.4.59.3:37021)
0620 09:21:24.369545 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:24.369579 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:24.370563 (+ 984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:24.370809 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:24.370812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:24.371048 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:24.371238 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:24.371242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:24.371862 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:24.371863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:24.372633 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:24.372635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:24.372940 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:24.372941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:24.373031 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:24.373657 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:24.373665 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:24.374945 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:24.376259 (+ 1314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:24.376264 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:24.376265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:24.376464 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:24.376631 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:24.376631 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:24.376631 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:24.376723 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:24.376944 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:24.376950 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:24.377059 (+ 109us) client_negotiation.cc:769] Sending connection context
0620 09:21:24.377171 (+ 112us) client_negotiation.cc:240] Negotiation successful
0620 09:21:24.377281 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260620 09:21:24.378003 8367 heartbeater.cc:347] Connected to a master server at 127.4.59.62:36677
I20260620 09:21:24.378104 8367 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:24.378274 8367 heartbeater.cc:511] Master 127.4.59.62:36677 requested a full tablet report, sending...
I20260620 09:21:24.378322 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 8238
I20260620 09:21:24.378446 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:21:24.378646 4332 external_mini_cluster.cc:1428] Setting key 622caa6d6b4c1103d8a0a2ec33d5f9a5
I20260620 09:21:24.378659 7908 ts_manager.cc:194] Registered new tserver with Master: 762c777210ef4312b983492b8791099e (127.4.59.3:42299)
I20260620 09:21:24.379132 7908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:37021
I20260620 09:21:24.379484 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 09:21:24 dist-test-slave-414w krb5kdc[7863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 09:21:24 dist-test-slave-414w krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.1@KRBTEST.COM
I20260620 09:21:24.402102 8374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:24.394487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34112 (local address 127.4.59.1:37071)
0620 09:21:24.394640 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:24.394644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:24.394654 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:21:24.394677 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:24.394678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:24.394691 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:24.394754 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:24.395270 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:24.395981 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:24.396670 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:24.396978 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:24.397032 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:24.399217 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:24.399224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:24.399224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:24.399230 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:24.400804 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:24.401243 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:24.401246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:24.401247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:24.401300 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:24.401536 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:24.401537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:24.401538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:24.401676 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:24.401862 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:24.401905 (+ 43us) server_negotiation.cc:299] Negotiation successful
0620 09:21:24.401982 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
Jun 20 09:21:24 dist-test-slave-414w krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:24.409814 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:24.403298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46506 (local address 127.4.59.62:36677)
0620 09:21:24.403351 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:24.403354 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:24.403399 (+ 45us) server_negotiation.cc:407] Connection header received
0620 09:21:24.403473 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:24.403474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:24.403483 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:24.403561 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:24.404000 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:24.404483 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:24.405167 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:24.405363 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:24.405445 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:24.407500 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:24.407501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:24.407501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:24.407507 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:24.408842 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:24.409129 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:24.409130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:24.409130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:24.409227 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:24.409385 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:24.409385 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:24.409385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:24.409508 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:24.409575 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:24.409652 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 09:21:24.409707 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260620 09:21:24.410044 7907 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46506
I20260620 09:21:24.410310 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 7970
I20260620 09:21:24.415576 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 8104
I20260620 09:21:24.419896 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 8238
I20260620 09:21:24.423862 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 7876
2026-06-20T09:21:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4837 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:21:24 dist-test-slave-414w krb5kdc[8379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:21:24 dist-test-slave-414w krb5kdc[8379](info): set up 2 sockets
Jun 20 09:21:24 dist-test-slave-414w krb5kdc[8379](info): commencing operation
krb5kdc: starting...
W20260620 09:21:25.485548 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:21:25 dist-test-slave-414w krb5kdc[8379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:21:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:21:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:21:25.600499 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:39685
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:39685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:25.679435 8392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:25.679651 8392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:25.679687 8392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:25.680819 8392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:25.680889 8392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:25.680914 8392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:25.680954 8392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:25.680979 8392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:21:25.682446 8392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:39685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:39685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:25.682716 8392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:25.682922 8392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:25.687608 8400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:25.687587 8398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:25.687649 8392 server_base.cc:1034] running on GCE node
W20260620 09:21:25.687578 8397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:25.688174 8392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:25.688483 8392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:25.689689 8392 hybrid_clock.cc:648] HybridClock initialized: now 1781947285689679 us; error 36 us; skew 500 ppm
Jun 20 09:21:25 dist-test-slave-414w krb5kdc[8379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947285, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:25.692302 8392 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:21:25.692663 8392 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:25.692739 8392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:25.692888 8392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:25.694002 8392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "f6922b840cfc4fdabc4d5be52fbf9017"
format_stamp: "Formatted at 2026-06-20 09:21:25 on dist-test-slave-414w"
server_key: "75fbee7ac14a39ea8e1c1376eacb139b"
server_key_iv: "b9834c359458fbb8866e7035e7cfe94f"
server_key_version: "encryptionkey@0"
I20260620 09:21:25.694401 8392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "f6922b840cfc4fdabc4d5be52fbf9017"
format_stamp: "Formatted at 2026-06-20 09:21:25 on dist-test-slave-414w"
server_key: "75fbee7ac14a39ea8e1c1376eacb139b"
server_key_iv: "b9834c359458fbb8866e7035e7cfe94f"
server_key_version: "encryptionkey@0"
I20260620 09:21:25.695787 8392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 09:21:25.696527 8406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:25.696682 8392 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:21:25.696787 8392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "f6922b840cfc4fdabc4d5be52fbf9017"
format_stamp: "Formatted at 2026-06-20 09:21:25 on dist-test-slave-414w"
server_key: "75fbee7ac14a39ea8e1c1376eacb139b"
server_key_iv: "b9834c359458fbb8866e7035e7cfe94f"
server_key_version: "encryptionkey@0"
I20260620 09:21:25.696864 8392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:25.736549 8392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:25.738979 8392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:25.739149 8392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:25.743175 8392 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:39685
I20260620 09:21:25.743212 8458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:39685 every 8 connection(s)
I20260620 09:21:25.743512 8392 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:21:25.744397 8459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:25.745098 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 8392
I20260620 09:21:25.745234 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:21:25.745419 4332 external_mini_cluster.cc:1428] Setting key 5fd1c450eb6013c0a436395cc0e139b1
I20260620 09:21:25.747537 8459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017: Bootstrap starting.
I20260620 09:21:25.748118 8459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:25.748389 8459 log.cc:824] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:25.749089 8459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017: No bootstrap required, opened a new log
I20260620 09:21:25.751555 8459 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6922b840cfc4fdabc4d5be52fbf9017" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39685 } }
I20260620 09:21:25.751708 8459 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:25.751732 8459 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6922b840cfc4fdabc4d5be52fbf9017, State: Initialized, Role: FOLLOWER
I20260620 09:21:25.751858 8459 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6922b840cfc4fdabc4d5be52fbf9017" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39685 } }
I20260620 09:21:25.751930 8459 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:21:25.751979 8459 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:21:25.752024 8459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [term 0 FOLLOWER]: Advancing to term 1
Jun 20 09:21:25 dist-test-slave-414w krb5kdc[8379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:25.752583 8459 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6922b840cfc4fdabc4d5be52fbf9017" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39685 } }
I20260620 09:21:25.752702 8459 leader_election.cc:302] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6922b840cfc4fdabc4d5be52fbf9017; no voters:
I20260620 09:21:25.752854 8459 leader_election.cc:288] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:21:25.752925 8464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:25.753058 8464 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [term 1 LEADER]: Becoming Leader. State: Replica: f6922b840cfc4fdabc4d5be52fbf9017, State: Running, Role: LEADER
I20260620 09:21:25.753096 8459 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:25.753170 8464 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6922b840cfc4fdabc4d5be52fbf9017" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39685 } }
I20260620 09:21:25.753484 8464 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6922b840cfc4fdabc4d5be52fbf9017. Latest consensus state: current_term: 1 leader_uuid: "f6922b840cfc4fdabc4d5be52fbf9017" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6922b840cfc4fdabc4d5be52fbf9017" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39685 } } }
I20260620 09:21:25.753584 8464 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:25.753569 8465 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6922b840cfc4fdabc4d5be52fbf9017" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6922b840cfc4fdabc4d5be52fbf9017" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39685 } } }
I20260620 09:21:25.753852 8472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:21:25.754493 8472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:21:25.753716 8465 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:25.755978 8472 catalog_manager.cc:1347] Generated new cluster ID: eda7f3aa9ecb4a3a816176719154eb12
I20260620 09:21:25.756037 8472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:21:25.756553 8462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:25.745928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60776 (local address 127.4.59.62:39685)
0620 09:21:25.746082 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:25.746091 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:25.746125 (+ 34us) server_negotiation.cc:407] Connection header received
0620 09:21:25.746525 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:25.746532 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:25.746575 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:25.746685 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:25.747455 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:25.748490 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:25.749527 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:25.749912 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:25.749993 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:25.752660 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:25.752664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:25.752664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:25.752674 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:25.754819 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:25.755216 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:25.755219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:25.755219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:25.755307 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:25.755545 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:25.755546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:25.755546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:25.755846 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:25.755982 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:25.756236 (+ 254us) server_negotiation.cc:299] Negotiation successful
0620 09:21:25.756368 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260620 09:21:25.765971 8472 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:21:25.766477 8472 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:21:25.769950 8472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017: Generated new TSK 0
I20260620 09:21:25.770107 8472 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:21:25.801585 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:39685
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:25.901896 8485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:25.902081 8485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:25.902105 8485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:25.903221 8485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:25.903261 8485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:25.903287 8485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:21:25.904749 8485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:39685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:25.904985 8485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:25.905267 8485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:25.910823 8491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:25.910852 8490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:25.910818 8493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:25.911018 8485 server_base.cc:1034] running on GCE node
I20260620 09:21:25.911247 8485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:25.911444 8485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:25.912580 8485 hybrid_clock.cc:648] HybridClock initialized: now 1781947285912562 us; error 35 us; skew 500 ppm
Jun 20 09:21:25 dist-test-slave-414w krb5kdc[8379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947285, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:25.915186 8485 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:21:25.915552 8485 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:25.915613 8485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:25.915704 8485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:25.916684 8485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "bc47045991104c309c296ac4eb5c2f5a"
format_stamp: "Formatted at 2026-06-20 09:21:25 on dist-test-slave-414w"
server_key: "223412d9e60a723dd3486127b18ecbaa"
server_key_iv: "ab3a9ced885f8b6a157e1b67f3556d23"
server_key_version: "encryptionkey@0"
I20260620 09:21:25.916970 8485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "bc47045991104c309c296ac4eb5c2f5a"
format_stamp: "Formatted at 2026-06-20 09:21:25 on dist-test-slave-414w"
server_key: "223412d9e60a723dd3486127b18ecbaa"
server_key_iv: "ab3a9ced885f8b6a157e1b67f3556d23"
server_key_version: "encryptionkey@0"
I20260620 09:21:25.918226 8485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260620 09:21:25.918990 8499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:25.919243 8485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:25.919355 8485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "bc47045991104c309c296ac4eb5c2f5a"
format_stamp: "Formatted at 2026-06-20 09:21:25 on dist-test-slave-414w"
server_key: "223412d9e60a723dd3486127b18ecbaa"
server_key_iv: "ab3a9ced885f8b6a157e1b67f3556d23"
server_key_version: "encryptionkey@0"
I20260620 09:21:25.919453 8485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:25.939137 8485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:25.941257 8485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:25.941431 8485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:25.941668 8485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:25.941978 8485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:25.942034 8485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:25.942078 8485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:25.942106 8485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:25.948053 8485 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:42247
I20260620 09:21:25.948098 8612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:42247 every 8 connection(s)
I20260620 09:21:25.948379 8485 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
Jun 20 09:21:25 dist-test-slave-414w krb5kdc[8379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947285, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:25.956969 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 8485
I20260620 09:21:25.957094 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:21:25.957321 4332 external_mini_cluster.cc:1428] Setting key 081e38f3cc205817f9624b0d9ba4e180
I20260620 09:21:25.957795 8462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:25.949175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:42223 (local address 127.4.59.62:39685)
0620 09:21:25.949226 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:25.949229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:25.949419 (+ 190us) server_negotiation.cc:407] Connection header received
0620 09:21:25.949940 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:25.949942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:25.949954 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:25.950005 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:25.950904 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:25.951412 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:25.952406 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:25.952613 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:25.952698 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:25.955023 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:25.955024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:25.955024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:25.955031 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:25.956322 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:25.956716 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:25.956717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:25.956717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:25.956766 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:25.957027 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:25.957028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:25.957028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:25.957220 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:25.957296 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:25.957614 (+ 318us) server_negotiation.cc:299] Negotiation successful
0620 09:21:25.957705 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 09:21:25.957958 8615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:25.949118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:39685 (local address 127.4.59.1:42223)
0620 09:21:25.949297 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:25.949333 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:25.949828 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:25.950072 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:25.950076 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:25.950326 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:25.950806 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:25.950810 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:25.951517 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:25.951518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:25.952305 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:25.952307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:25.952709 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:25.952710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:25.952825 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:25.953380 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:25.953388 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:25.954925 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:25.956402 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:25.956408 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:25.956409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:25.956649 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:25.956861 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:25.956861 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:25.956861 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:25.956956 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:25.957295 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:25.957300 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:25.957441 (+ 141us) client_negotiation.cc:769] Sending connection context
0620 09:21:25.957583 (+ 142us) client_negotiation.cc:240] Negotiation successful
0620 09:21:25.957734 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
I20260620 09:21:25.958547 8613 heartbeater.cc:347] Connected to a master server at 127.4.59.62:39685
I20260620 09:21:25.958657 8613 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:25.958809 8613 heartbeater.cc:511] Master 127.4.59.62:39685 requested a full tablet report, sending...
I20260620 09:21:25.959342 8423 ts_manager.cc:194] Registered new tserver with Master: bc47045991104c309c296ac4eb5c2f5a (127.4.59.1:42247)
I20260620 09:21:25.960328 8423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:42223
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:21:25.983646 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:39685
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:26.066040 8618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:26.066236 8618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:26.066258 8618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:26.067396 8618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:26.067437 8618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:26.067464 8618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:21:26.068992 8618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:39685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:26.069265 8618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:26.069501 8618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:26.074317 8623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:26.074317 8624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:26.074465 8626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:26.074604 8618 server_base.cc:1034] running on GCE node
I20260620 09:21:26.074841 8618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:26.075155 8618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:26.076339 8618 hybrid_clock.cc:648] HybridClock initialized: now 1781947286076326 us; error 31 us; skew 500 ppm
Jun 20 09:21:26 dist-test-slave-414w krb5kdc[8379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947286, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:26.079134 8618 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:21:26.079514 8618 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:26.079587 8618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:26.079733 8618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:26.080792 8618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "63f6278d85b045a69c2f54a4c2aae661"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "91ff2ecdc5bd33cbf53b6148a1a8642b"
server_key_iv: "46d8c6cc95809b4be8e1133ef47f137e"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.081212 8618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "63f6278d85b045a69c2f54a4c2aae661"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "91ff2ecdc5bd33cbf53b6148a1a8642b"
server_key_iv: "46d8c6cc95809b4be8e1133ef47f137e"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.082517 8618 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:26.083238 8632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:26.083544 8618 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:26.083642 8618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "63f6278d85b045a69c2f54a4c2aae661"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "91ff2ecdc5bd33cbf53b6148a1a8642b"
server_key_iv: "46d8c6cc95809b4be8e1133ef47f137e"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.083729 8618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:26.106554 8618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:26.108512 8618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:26.108621 8618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:26.108804 8618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:26.109124 8618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:26.109212 8618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:26.109265 8618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:26.109293 8618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:26.115294 8618 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:43377
I20260620 09:21:26.115383 8745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:43377 every 8 connection(s)
I20260620 09:21:26.115653 8618 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:21:26.118640 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 8618
I20260620 09:21:26.118768 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:21:26.118937 4332 external_mini_cluster.cc:1428] Setting key bbd504e7ef9719e1df114b628b824e01
Jun 20 09:21:26 dist-test-slave-414w krb5kdc[8379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947286, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:26.126041 8748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.116311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:39685 (local address 127.4.59.2:60631)
0620 09:21:26.116481 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:26.116521 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:26.117016 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:26.117278 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:26.117282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:26.117544 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:26.118017 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:26.118022 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.118729 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.118730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:26.119801 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:26.119804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.120189 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.120190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:26.120322 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:26.120878 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:26.120888 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:26.122733 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:26.124685 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:26.124690 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:26.124691 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:26.124895 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:26.125121 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:26.125122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:26.125122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:26.125277 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:26.125577 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:26.125582 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:26.125696 (+ 114us) client_negotiation.cc:769] Sending connection context
0620 09:21:26.125773 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 09:21:26.125884 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260620 09:21:26.126035 8462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.116337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:60631 (local address 127.4.59.62:39685)
0620 09:21:26.116397 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.116398 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.116601 (+ 203us) server_negotiation.cc:407] Connection header received
0620 09:21:26.117120 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.117121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.117135 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.117207 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:26.118120 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.118620 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.119893 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.120104 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.120182 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:26.122826 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:26.122827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:26.122828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:26.122836 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:26.124593 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:26.124974 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:26.124975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:26.124976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:26.125041 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:26.125352 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:26.125353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:26.125353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:26.125500 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:26.125576 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.125824 (+ 248us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.125929 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 09:21:26.126513 8746 heartbeater.cc:347] Connected to a master server at 127.4.59.62:39685
I20260620 09:21:26.126616 8746 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:26.126775 8746 heartbeater.cc:511] Master 127.4.59.62:39685 requested a full tablet report, sending...
I20260620 09:21:26.127158 8423 ts_manager.cc:194] Registered new tserver with Master: 63f6278d85b045a69c2f54a4c2aae661 (127.4.59.2:43377)
I20260620 09:21:26.127534 8423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:60631
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:21:26.144215 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:39685
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:26.223208 8751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:26.223424 8751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:26.223460 8751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:26.224599 8751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:26.224665 8751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:26.224712 8751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:21:26.226230 8751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:39685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:26.226500 8751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:26.226732 8751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:26.231896 8759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:26.231896 8757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:26.231900 8756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:26.232460 8751 server_base.cc:1034] running on GCE node
I20260620 09:21:26.232750 8751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:26.232998 8751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:26.234158 8751 hybrid_clock.cc:648] HybridClock initialized: now 1781947286234135 us; error 39 us; skew 500 ppm
Jun 20 09:21:26 dist-test-slave-414w krb5kdc[8379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947286, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:26.236799 8751 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:21:26.237272 8751 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:26.237323 8751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:26.237452 8751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:26.238525 8751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "6996fbf542b345389a594d133dd3809a"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "ae7d862ac160df517f46b870178fb884"
server_key_iv: "42136d55042701dd22066d5241346727"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.238881 8751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "6996fbf542b345389a594d133dd3809a"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "ae7d862ac160df517f46b870178fb884"
server_key_iv: "42136d55042701dd22066d5241346727"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.240207 8751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:21:26.240958 8765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:26.241225 8751 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:21:26.241317 8751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "6996fbf542b345389a594d133dd3809a"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "ae7d862ac160df517f46b870178fb884"
server_key_iv: "42136d55042701dd22066d5241346727"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.241391 8751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:26.266896 8751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:26.269443 8751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:26.269600 8751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:26.269865 8751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:26.270229 8751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:26.270265 8751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:26.270310 8751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:26.270332 8751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:26.275899 8751 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:41081
I20260620 09:21:26.275974 8878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:41081 every 8 connection(s)
I20260620 09:21:26.276242 8751 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:21:26.279125 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 8751
I20260620 09:21:26.279209 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:21:26.279389 4332 external_mini_cluster.cc:1428] Setting key 8457ac00eb4af57b556c925a3da592ae
Jun 20 09:21:26 dist-test-slave-414w krb5kdc[8379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947286, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:26.286525 8462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.277107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:32823 (local address 127.4.59.62:39685)
0620 09:21:26.277232 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.277234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.277494 (+ 260us) server_negotiation.cc:407] Connection header received
0620 09:21:26.278018 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.278020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.278031 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.278081 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:26.279090 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.279943 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.281033 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.281289 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.281344 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:26.283399 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:26.283400 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:26.283401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:26.283407 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:26.284943 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:26.285587 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:26.285588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:26.285588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:26.285646 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:26.285875 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:26.285876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:26.285876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:26.285990 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:26.286041 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.286360 (+ 319us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.286460 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260620 09:21:26.286741 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.277027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:39685 (local address 127.4.59.3:32823)
0620 09:21:26.277371 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:26.277406 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:26.277913 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:26.278183 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:26.278187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:26.278450 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:26.278977 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:26.278983 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.280043 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.280044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:26.280959 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:26.280961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.281363 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.281364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:26.281465 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:26.281914 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:26.281922 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:26.283302 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:26.285031 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:26.285036 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:26.285038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:26.285495 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:26.285715 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:26.285716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:26.285716 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:26.285816 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:26.286046 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:26.286050 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:26.286181 (+ 131us) client_negotiation.cc:769] Sending connection context
0620 09:21:26.286264 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 09:21:26.286472 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":267,"threads_started":1}
I20260620 09:21:26.287197 8879 heartbeater.cc:347] Connected to a master server at 127.4.59.62:39685
I20260620 09:21:26.287292 8879 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:26.287500 8879 heartbeater.cc:511] Master 127.4.59.62:39685 requested a full tablet report, sending...
I20260620 09:21:26.287884 8423 ts_manager.cc:194] Registered new tserver with Master: 6996fbf542b345389a594d133dd3809a (127.4.59.3:41081)
I20260620 09:21:26.288372 8423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:32823
I20260620 09:21:26.290972 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:21:26.298074 8462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.292337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60790 (local address 127.4.59.62:39685)
0620 09:21:26.292405 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.292407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.292415 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:21:26.292435 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.292436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.292445 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.292494 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:26.293026 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.293566 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.294282 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.294482 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.294564 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:26.295633 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:26.295634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:26.295634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:26.295640 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:26.296884 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:26.297263 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:26.297264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:26.297265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:26.297324 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:26.297539 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:26.297540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:26.297540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:26.297684 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:26.297788 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.297893 (+ 105us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.297968 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 09:21:26.299754 8423 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:21:26.301517 8423 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:21:26.307849 8890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.303646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:42247 (local address 127.0.0.1:57178)
0620 09:21:26.303920 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:26.303965 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:26.304077 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:26.304348 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:26.304351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:26.304354 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:26.304425 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:26.304426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.305545 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.305546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:26.306827 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:26.306829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.307551 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.307552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:26.307628 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:26.307642 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 09:21:26.307711 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 09:21:26.307764 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":37,"thread_start_us":50,"threads_started":1}
I20260620 09:21:26.307951 8894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.303836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:41081 (local address 127.0.0.1:34738)
0620 09:21:26.304221 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:26.304230 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:26.304274 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:26.304466 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:26.304467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:26.304469 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:26.304503 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:26.304504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.305947 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.305948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:26.306854 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:26.306856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.307699 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.307700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:26.307752 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:26.307755 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:21:26.307812 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 09:21:26.307843 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":66,"threads_started":1}
I20260620 09:21:26.307963 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.303917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34738 (local address 127.4.59.3:41081)
0620 09:21:26.304120 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.304124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.304330 (+ 206us) server_negotiation.cc:407] Connection header received
0620 09:21:26.304374 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.304375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.304390 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.304658 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:26.304812 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.305838 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.306918 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.307625 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.307701 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:26.307758 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.307828 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.307867 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":82,"threads_started":1}
I20260620 09:21:26.308226 8891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.303692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:43377 (local address 127.0.0.1:36774)
0620 09:21:26.304101 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:26.304116 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:26.304213 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:26.304473 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:26.304474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:26.304477 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:26.304530 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:26.304531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.305828 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.305829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:26.306958 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:26.306960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.307836 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.307837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:26.307895 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:26.307899 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:21:26.308117 (+ 218us) client_negotiation.cc:240] Negotiation successful
0620 09:21:26.308164 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":85,"threads_started":1}
I20260620 09:21:26.308274 8893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.303787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36774 (local address 127.4.59.2:43377)
0620 09:21:26.303961 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.303967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.304231 (+ 264us) server_negotiation.cc:407] Connection header received
0620 09:21:26.304278 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.304281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.304297 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.304373 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:26.304615 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.305727 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.307030 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.307697 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.307754 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:26.307800 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.308154 (+ 354us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.308194 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":76,"threads_started":1}
I20260620 09:21:26.308781 8892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.303749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57178 (local address 127.4.59.1:42247)
0620 09:21:26.304044 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.304047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.304059 (+ 12us) server_negotiation.cc:407] Connection header received
0620 09:21:26.304194 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.304196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.304214 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.304343 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:26.304508 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.305442 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.306927 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.307448 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.307549 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:26.307620 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.308660 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.308707 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":73,"threads_started":1}
I20260620 09:21:26.308673 8813 tablet_service.cc:1467] Processing CreateTablet for tablet eea561d985b145cba13762dcc335890a (DEFAULT_TABLE table=test-table [id=7371f716f66444a78d925af5c356a543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:26.309033 8813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eea561d985b145cba13762dcc335890a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:26.308960 8680 tablet_service.cc:1467] Processing CreateTablet for tablet eea561d985b145cba13762dcc335890a (DEFAULT_TABLE table=test-table [id=7371f716f66444a78d925af5c356a543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:26.309332 8680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eea561d985b145cba13762dcc335890a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:26.309544 8547 tablet_service.cc:1467] Processing CreateTablet for tablet eea561d985b145cba13762dcc335890a (DEFAULT_TABLE table=test-table [id=7371f716f66444a78d925af5c356a543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:26.309839 8547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eea561d985b145cba13762dcc335890a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:26.311888 8896 tablet_bootstrap.cc:492] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a: Bootstrap starting.
I20260620 09:21:26.312465 8896 tablet_bootstrap.cc:654] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:26.312615 8897 tablet_bootstrap.cc:492] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661: Bootstrap starting.
I20260620 09:21:26.312968 8898 tablet_bootstrap.cc:492] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a: Bootstrap starting.
I20260620 09:21:26.313107 8896 log.cc:824] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:26.313405 8897 tablet_bootstrap.cc:654] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:26.313643 8898 tablet_bootstrap.cc:654] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:26.313999 8897 log.cc:824] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:26.314028 8896 tablet_bootstrap.cc:492] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a: No bootstrap required, opened a new log
I20260620 09:21:26.314138 8896 ts_tablet_manager.cc:1397] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:21:26.314293 8898 log.cc:824] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:26.314918 8897 tablet_bootstrap.cc:492] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661: No bootstrap required, opened a new log
I20260620 09:21:26.314990 8897 ts_tablet_manager.cc:1397] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 09:21:26.315150 8898 tablet_bootstrap.cc:492] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a: No bootstrap required, opened a new log
I20260620 09:21:26.315241 8898 ts_tablet_manager.cc:1397] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:21:26.315908 8896 raft_consensus.cc:348] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc47045991104c309c296ac4eb5c2f5a" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 42247 } } peers { permanent_uuid: "6996fbf542b345389a594d133dd3809a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41081 } } peers { permanent_uuid: "63f6278d85b045a69c2f54a4c2aae661" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43377 } }
I20260620 09:21:26.316061 8896 raft_consensus.cc:374] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:26.316113 8896 raft_consensus.cc:729] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6996fbf542b345389a594d133dd3809a, State: Initialized, Role: FOLLOWER
I20260620 09:21:26.316226 8896 consensus_queue.cc:260] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc47045991104c309c296ac4eb5c2f5a" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 42247 } } peers { permanent_uuid: "6996fbf542b345389a594d133dd3809a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41081 } } peers { permanent_uuid: "63f6278d85b045a69c2f54a4c2aae661" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43377 } }
I20260620 09:21:26.316509 8896 ts_tablet_manager.cc:1428] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:21:26.316645 8879 heartbeater.cc:503] Master 127.4.59.62:39685 was elected leader, sending a full tablet report...
I20260620 09:21:26.316943 8897 raft_consensus.cc:348] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc47045991104c309c296ac4eb5c2f5a" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 42247 } } peers { permanent_uuid: "6996fbf542b345389a594d133dd3809a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41081 } } peers { permanent_uuid: "63f6278d85b045a69c2f54a4c2aae661" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43377 } }
I20260620 09:21:26.317095 8897 raft_consensus.cc:374] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:26.317159 8897 raft_consensus.cc:729] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63f6278d85b045a69c2f54a4c2aae661, State: Initialized, Role: FOLLOWER
I20260620 09:21:26.317265 8897 consensus_queue.cc:260] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc47045991104c309c296ac4eb5c2f5a" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 42247 } } peers { permanent_uuid: "6996fbf542b345389a594d133dd3809a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41081 } } peers { permanent_uuid: "63f6278d85b045a69c2f54a4c2aae661" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43377 } }
I20260620 09:21:26.317538 8897 ts_tablet_manager.cc:1428] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:21:26.317710 8746 heartbeater.cc:503] Master 127.4.59.62:39685 was elected leader, sending a full tablet report...
I20260620 09:21:26.317716 8898 raft_consensus.cc:348] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc47045991104c309c296ac4eb5c2f5a" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 42247 } } peers { permanent_uuid: "6996fbf542b345389a594d133dd3809a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41081 } } peers { permanent_uuid: "63f6278d85b045a69c2f54a4c2aae661" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43377 } }
I20260620 09:21:26.317878 8898 raft_consensus.cc:374] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:26.317920 8898 raft_consensus.cc:729] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc47045991104c309c296ac4eb5c2f5a, State: Initialized, Role: FOLLOWER
I20260620 09:21:26.318023 8898 consensus_queue.cc:260] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc47045991104c309c296ac4eb5c2f5a" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 42247 } } peers { permanent_uuid: "6996fbf542b345389a594d133dd3809a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41081 } } peers { permanent_uuid: "63f6278d85b045a69c2f54a4c2aae661" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43377 } }
I20260620 09:21:26.318291 8898 ts_tablet_manager.cc:1428] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:21:26.318434 8613 heartbeater.cc:503] Master 127.4.59.62:39685 was elected leader, sending a full tablet report...
W20260620 09:21:26.366200 8747 tablet.cc:2369] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:21:26.367003 8903 raft_consensus.cc:482] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:21:26.367093 8903 raft_consensus.cc:504] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc47045991104c309c296ac4eb5c2f5a" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 42247 } } peers { permanent_uuid: "6996fbf542b345389a594d133dd3809a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41081 } } peers { permanent_uuid: "63f6278d85b045a69c2f54a4c2aae661" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43377 } }
I20260620 09:21:26.367425 8903 leader_election.cc:288] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc47045991104c309c296ac4eb5c2f5a (127.4.59.1:42247), 6996fbf542b345389a594d133dd3809a (127.4.59.3:41081)
I20260620 09:21:26.370220 8892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.367700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:38303 (local address 127.4.59.1:42247)
0620 09:21:26.367774 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.367777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.367787 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:21:26.367817 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.367818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.367831 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.367897 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:26.368078 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.368558 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.369529 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.369871 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.369953 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:26.369962 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.370120 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.370160 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 09:21:26.370213 8748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.367603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:42247 (local address 127.4.59.2:38303)
0620 09:21:26.367668 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:26.367681 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:26.367725 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:26.367906 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:26.367907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:26.367909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:26.367971 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:26.367973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.368642 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.368644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:26.369408 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:26.369411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.369951 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.369952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:26.370034 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:26.370037 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:21:26.370102 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 09:21:26.370144 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 09:21:26.370483 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.367801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:50259 (local address 127.4.59.3:41081)
0620 09:21:26.367851 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.367853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.367954 (+ 101us) server_negotiation.cc:407] Connection header received
0620 09:21:26.368025 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.368026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.368039 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.368083 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:26.368258 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.368710 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.369704 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.370085 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.370167 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:26.370177 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.370365 (+ 188us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.370416 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 09:21:26.370546 8567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eea561d985b145cba13762dcc335890a" candidate_uuid: "63f6278d85b045a69c2f54a4c2aae661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc47045991104c309c296ac4eb5c2f5a" is_pre_election: true
I20260620 09:21:26.370682 8905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.367728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:41081 (local address 127.4.59.2:50259)
0620 09:21:26.367899 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:26.367910 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:26.367963 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:26.368084 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:26.368084 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:26.368085 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:26.368184 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:26.368186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.368794 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.368795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:26.369616 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:26.369619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.370180 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.370181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:26.370253 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:26.370256 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:21:26.370577 (+ 321us) client_negotiation.cc:240] Negotiation successful
0620 09:21:26.370616 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":113,"threads_started":1}
I20260620 09:21:26.370702 8567 raft_consensus.cc:2454] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63f6278d85b045a69c2f54a4c2aae661 in term 0.
I20260620 09:21:26.370942 8635 leader_election.cc:302] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63f6278d85b045a69c2f54a4c2aae661, bc47045991104c309c296ac4eb5c2f5a; no voters:
I20260620 09:21:26.371048 8833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eea561d985b145cba13762dcc335890a" candidate_uuid: "63f6278d85b045a69c2f54a4c2aae661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6996fbf542b345389a594d133dd3809a" is_pre_election: true
I20260620 09:21:26.371117 8903 raft_consensus.cc:2781] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:21:26.371156 8903 raft_consensus.cc:482] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:21:26.371171 8903 raft_consensus.cc:3037] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:26.371209 8833 raft_consensus.cc:2454] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63f6278d85b045a69c2f54a4c2aae661 in term 0.
I20260620 09:21:26.371919 8903 raft_consensus.cc:504] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc47045991104c309c296ac4eb5c2f5a" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 42247 } } peers { permanent_uuid: "6996fbf542b345389a594d133dd3809a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41081 } } peers { permanent_uuid: "63f6278d85b045a69c2f54a4c2aae661" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43377 } }
I20260620 09:21:26.372051 8903 leader_election.cc:288] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [CANDIDATE]: Term 1 election: Requested vote from peers bc47045991104c309c296ac4eb5c2f5a (127.4.59.1:42247), 6996fbf542b345389a594d133dd3809a (127.4.59.3:41081)
I20260620 09:21:26.372236 8567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eea561d985b145cba13762dcc335890a" candidate_uuid: "63f6278d85b045a69c2f54a4c2aae661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc47045991104c309c296ac4eb5c2f5a"
I20260620 09:21:26.372311 8567 raft_consensus.cc:3037] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:26.372264 8833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eea561d985b145cba13762dcc335890a" candidate_uuid: "63f6278d85b045a69c2f54a4c2aae661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6996fbf542b345389a594d133dd3809a"
I20260620 09:21:26.372372 8833 raft_consensus.cc:3037] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:26.373067 8833 raft_consensus.cc:2454] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63f6278d85b045a69c2f54a4c2aae661 in term 1.
I20260620 09:21:26.373080 8567 raft_consensus.cc:2454] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63f6278d85b045a69c2f54a4c2aae661 in term 1.
I20260620 09:21:26.373237 8635 leader_election.cc:302] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63f6278d85b045a69c2f54a4c2aae661, 6996fbf542b345389a594d133dd3809a; no voters:
I20260620 09:21:26.373327 8903 raft_consensus.cc:2781] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:26.373494 8903 raft_consensus.cc:686] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [term 1 LEADER]: Becoming Leader. State: Replica: 63f6278d85b045a69c2f54a4c2aae661, State: Running, Role: LEADER
I20260620 09:21:26.373600 8903 consensus_queue.cc:237] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc47045991104c309c296ac4eb5c2f5a" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 42247 } } peers { permanent_uuid: "6996fbf542b345389a594d133dd3809a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41081 } } peers { permanent_uuid: "63f6278d85b045a69c2f54a4c2aae661" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43377 } }
I20260620 09:21:26.374548 8422 catalog_manager.cc:5495] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 reported cstate change: term changed from 0 to 1, leader changed from <none> to 63f6278d85b045a69c2f54a4c2aae661 (127.4.59.2). New cstate: current_term: 1 leader_uuid: "63f6278d85b045a69c2f54a4c2aae661" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc47045991104c309c296ac4eb5c2f5a" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6996fbf542b345389a594d133dd3809a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63f6278d85b045a69c2f54a4c2aae661" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43377 } health_report { overall_health: HEALTHY } } }
I20260620 09:21:26.380203 8893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.377859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36788 (local address 127.4.59.2:43377)
0620 09:21:26.377903 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.377905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.377914 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:21:26.377987 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.377987 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.378005 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.378073 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:21:26.378194 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.378641 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.379395 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:26.379599 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:26.379686 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:26.379864 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:21:26.379958 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:21:26.380055 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.380086 (+ 31us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.380104 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 09:21:26.381758 8833 raft_consensus.cc:1261] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a [term 1 FOLLOWER]: Refusing update from remote peer 63f6278d85b045a69c2f54a4c2aae661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:26.381763 8567 raft_consensus.cc:1261] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a [term 1 FOLLOWER]: Refusing update from remote peer 63f6278d85b045a69c2f54a4c2aae661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:26.382066 8903 consensus_queue.cc:1035] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6996fbf542b345389a594d133dd3809a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:26.382182 8906 consensus_queue.cc:1035] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc47045991104c309c296ac4eb5c2f5a" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:26.385030 8909 mvcc.cc:205] Tried to move back new op lower bound from 7298856085017128960 to 7298856084986826752. Current Snapshot: MvccSnapshot[applied={T|T < 7298856085017128960}]
I20260620 09:21:26.385308 8911 mvcc.cc:205] Tried to move back new op lower bound from 7298856085017128960 to 7298856084986826752. Current Snapshot: MvccSnapshot[applied={T|T < 7298856085017128960}]
I20260620 09:21:26.385457 8910 mvcc.cc:205] Tried to move back new op lower bound from 7298856085017128960 to 7298856084986826752. Current Snapshot: MvccSnapshot[applied={T|T < 7298856085017128960}]
I20260620 09:21:26.387539 8423 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:26.387637 8423 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:26.388363 8423 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6922b840cfc4fdabc4d5be52fbf9017: Sending DeleteTablet for 3 replicas of tablet eea561d985b145cba13762dcc335890a
I20260620 09:21:26.388688 8680 tablet_service.cc:1514] Processing DeleteTablet for tablet eea561d985b145cba13762dcc335890a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:26 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:36774
I20260620 09:21:26.388710 8547 tablet_service.cc:1514] Processing DeleteTablet for tablet eea561d985b145cba13762dcc335890a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:26 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:57178
I20260620 09:21:26.388900 8920 tablet_replica.cc:330] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661: stopping tablet replica
I20260620 09:21:26.388893 8813 tablet_service.cc:1514] Processing DeleteTablet for tablet eea561d985b145cba13762dcc335890a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:26 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:34738
I20260620 09:21:26.388996 8920 raft_consensus.cc:2229] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:21:26.389107 8920 raft_consensus.cc:2258] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:26.389169 8921 tablet_replica.cc:330] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a: stopping tablet replica
I20260620 09:21:26.389322 8921 raft_consensus.cc:2229] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:26.389423 8921 raft_consensus.cc:2258] T eea561d985b145cba13762dcc335890a P bc47045991104c309c296ac4eb5c2f5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:26.389458 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 8485
I20260620 09:21:26.389523 8920 ts_tablet_manager.cc:1905] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:26.391475 8920 ts_tablet_manager.cc:1918] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:21:26.391525 8920 log.cc:1196] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/eea561d985b145cba13762dcc335890a
I20260620 09:21:26.391731 8920 ts_tablet_manager.cc:1939] T eea561d985b145cba13762dcc335890a P 63f6278d85b045a69c2f54a4c2aae661: Deleting consensus metadata
I20260620 09:21:26.393389 8408 catalog_manager.cc:4841] TS 63f6278d85b045a69c2f54a4c2aae661 (127.4.59.2:43377): tablet eea561d985b145cba13762dcc335890a (table test-table [id=7371f716f66444a78d925af5c356a543]) successfully deleted
I20260620 09:21:26.393954 8922 tablet_replica.cc:330] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a: stopping tablet replica
I20260620 09:21:26.394078 8922 raft_consensus.cc:2229] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:26.394192 8922 raft_consensus.cc:2258] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 09:21:26.395390 8409 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.59.1:42247 (error 108)
W20260620 09:21:26.395921 8891 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:21:26.395781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:42247 (local address 127.0.0.1:57190)
0620 09:21:26.395837 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:26.395877 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:42247: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 09:21:26.396029 8409 catalog_manager.cc:4584] TS bc47045991104c309c296ac4eb5c2f5a (127.4.59.1:42247): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eea561d985b145cba13762dcc335890a: Network error: Client connection negotiation failed: client connection to 127.4.59.1:42247: connect: Connection reset by peer (error 104)
I20260620 09:21:26.396041 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 8618
I20260620 09:21:26.400801 8922 ts_tablet_manager.cc:1905] T eea561d985b145cba13762dcc335890a P 6996fbf542b345389a594d133dd3809a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:26.400914 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 8751
I20260620 09:21:26.404816 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 8392
I20260620 09:21:26.404879 8891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.404745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:41081 (local address 127.0.0.1:34748)
0620 09:21:26.404825 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:26.404852 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.3:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
2026-06-20T09:21:26Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1981 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-20T09:21:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:21:26Z Disabled control of system clock
I20260620 09:21:26.432513 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:39195
--webserver_interface=127.4.59.62
--webserver_port=0
--builtin_ntp_servers=127.4.59.20:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:39195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 09:21:26.513224 8927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:26.513418 8927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:26.513439 8927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:26.514482 8927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:26.514516 8927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:26.514530 8927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:26.514544 8927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:26.514556 8927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:21:26.515820 8927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:39195
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:39195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:26.516057 8927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:26.516304 8927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:26.520849 8935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:26.520830 8932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:26.520895 8933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:26.521126 8927 server_base.cc:1034] running on GCE node
I20260620 09:21:26.521427 8927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:26.521664 8927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:26.522807 8927 hybrid_clock.cc:648] HybridClock initialized: now 1781947286522794 us; error 28 us; skew 500 ppm
I20260620 09:21:26.524174 8927 webserver.cc:460] Webserver started at http://127.4.59.62:38077/ using document root <none> and password file <none>
I20260620 09:21:26.524426 8927 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:26.524474 8927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:26.524627 8927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:26.525732 8927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "d45aa95057e54e93a6681dbacc4721d5"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "95d6eb20be542fb32943ddc7ea60594f"
server_key_iv: "c01ba62129b9c946f7750496043f9313"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.526086 8927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "d45aa95057e54e93a6681dbacc4721d5"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "95d6eb20be542fb32943ddc7ea60594f"
server_key_iv: "c01ba62129b9c946f7750496043f9313"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.527367 8927 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260620 09:21:26.528038 8941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:26.528280 8927 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:26.528405 8927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "d45aa95057e54e93a6681dbacc4721d5"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "95d6eb20be542fb32943ddc7ea60594f"
server_key_iv: "c01ba62129b9c946f7750496043f9313"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.528496 8927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:26.532022 8927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:26.532356 8927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:26.532534 8927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:26.536556 8927 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:39195
I20260620 09:21:26.536598 8993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:39195 every 8 connection(s)
I20260620 09:21:26.536927 8927 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:21:26.537572 8994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:26.540151 8994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5: Bootstrap starting.
I20260620 09:21:26.540650 8994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:26.540884 8994 log.cc:824] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:26.541517 8994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5: No bootstrap required, opened a new log
I20260620 09:21:26.542989 8994 raft_consensus.cc:348] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d45aa95057e54e93a6681dbacc4721d5" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39195 } }
I20260620 09:21:26.543126 8994 raft_consensus.cc:374] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:26.543149 8994 raft_consensus.cc:729] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d45aa95057e54e93a6681dbacc4721d5, State: Initialized, Role: FOLLOWER
I20260620 09:21:26.543278 8994 consensus_queue.cc:260] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d45aa95057e54e93a6681dbacc4721d5" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39195 } }
I20260620 09:21:26.543355 8994 raft_consensus.cc:388] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:21:26.543377 8994 raft_consensus.cc:482] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:21:26.543404 8994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:26.543866 8994 raft_consensus.cc:504] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d45aa95057e54e93a6681dbacc4721d5" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39195 } }
I20260620 09:21:26.543944 8994 leader_election.cc:302] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d45aa95057e54e93a6681dbacc4721d5; no voters:
I20260620 09:21:26.544075 8994 leader_election.cc:288] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:21:26.544194 8997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:26.544310 8994 sys_catalog.cc:564] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:26.544346 8997 raft_consensus.cc:686] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [term 1 LEADER]: Becoming Leader. State: Replica: d45aa95057e54e93a6681dbacc4721d5, State: Running, Role: LEADER
I20260620 09:21:26.544423 8997 consensus_queue.cc:237] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d45aa95057e54e93a6681dbacc4721d5" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39195 } }
I20260620 09:21:26.544817 8999 sys_catalog.cc:455] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d45aa95057e54e93a6681dbacc4721d5. Latest consensus state: current_term: 1 leader_uuid: "d45aa95057e54e93a6681dbacc4721d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d45aa95057e54e93a6681dbacc4721d5" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39195 } } }
I20260620 09:21:26.544903 8999 sys_catalog.cc:458] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:26.545190 8998 sys_catalog.cc:455] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d45aa95057e54e93a6681dbacc4721d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d45aa95057e54e93a6681dbacc4721d5" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39195 } } }
I20260620 09:21:26.545310 8998 sys_catalog.cc:458] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:26.545501 9006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:21:26.545981 9006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:21:26.546545 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 8927
I20260620 09:21:26.546662 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:21:26.546850 4332 external_mini_cluster.cc:1428] Setting key bffcc10a947e05990369f7edc04a7365
I20260620 09:21:26.547492 9006 catalog_manager.cc:1347] Generated new cluster ID: c23bccc5209b43d69afbf92f00c6e137
I20260620 09:21:26.547567 9006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:21:26.549271 9017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.547328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51350 (local address 127.4.59.62:39195)
0620 09:21:26.547813 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.547821 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.547851 (+ 30us) server_negotiation.cc:407] Connection header received
0620 09:21:26.547956 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.547963 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.548004 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.548142 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:26.548809 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:26.548812 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:26.548813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:21:26.548817 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:26.548861 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:21:26.548874 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:26.548945 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.549015 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.549073 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":68,"threads_started":1}
I20260620 09:21:26.556777 9006 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:21:26.556923 9006 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:21:26.564790 9006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5: Generated new TSK 0
I20260620 09:21:26.564980 9006 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:21:26.570748 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:39195
--builtin_ntp_servers=127.4.59.20:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 09:21:26.658334 9018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:26.658581 9018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:26.658619 9018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:26.659842 9018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:26.659904 9018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:26.659948 9018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:21:26.661363 9018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:39195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:26.661619 9018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:26.661906 9018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 09:21:26.667655 9018 server_base.cc:1034] running on GCE node
W20260620 09:21:26.667639 9024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:26.667793 9026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:26.667634 9023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:26.668028 9018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:26.668301 9018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:26.669450 9018 hybrid_clock.cc:648] HybridClock initialized: now 1781947286669437 us; error 28 us; skew 500 ppm
I20260620 09:21:26.670776 9018 webserver.cc:460] Webserver started at http://127.4.59.1:34873/ using document root <none> and password file <none>
I20260620 09:21:26.671015 9018 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:26.671083 9018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:26.671218 9018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:26.672262 9018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "7e038e0c75954b0b82051a4be44d7366"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "7f81658bcda3b7b59fdb09a9ae9d257e"
server_key_iv: "2b0bcd28b5ad6601b24f1e9c0bf1c428"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.672612 9018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "7e038e0c75954b0b82051a4be44d7366"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "7f81658bcda3b7b59fdb09a9ae9d257e"
server_key_iv: "2b0bcd28b5ad6601b24f1e9c0bf1c428"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.673992 9018 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:21:26.674751 9032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:26.675063 9018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:21:26.675161 9018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "7e038e0c75954b0b82051a4be44d7366"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "7f81658bcda3b7b59fdb09a9ae9d257e"
server_key_iv: "2b0bcd28b5ad6601b24f1e9c0bf1c428"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.675246 9018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:26.679684 9018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:26.679946 9018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:26.680086 9018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:26.680279 9018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:26.680575 9018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:26.680612 9018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:26.680639 9018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:26.680657 9018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:26.687446 9018 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:46423
I20260620 09:21:26.687531 9145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:46423 every 8 connection(s)
I20260620 09:21:26.687870 9018 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:21:26.691911 9017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.688645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:50083 (local address 127.4.59.62:39195)
0620 09:21:26.688723 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.688725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.688981 (+ 256us) server_negotiation.cc:407] Connection header received
0620 09:21:26.689638 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.689639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.689645 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.689716 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:26.691407 (+ 1691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:26.691410 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:26.691410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:21:26.691419 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:26.691457 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:21:26.691464 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:26.691587 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.691776 (+ 189us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.691814 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 09:21:26.692332 9148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.688622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:39195 (local address 127.4.59.1:50083)
0620 09:21:26.688864 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:26.688897 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:26.689543 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:26.689774 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:26.689778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:26.689785 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:26.691013 (+ 1228us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:21:26.691014 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:26.691120 (+ 106us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:21:26.691182 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:26.691636 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:26.691655 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:26.691660 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 09:21:26.691776 (+ 116us) client_negotiation.cc:240] Negotiation successful
0620 09:21:26.692092 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":106,"threads_started":1}
I20260620 09:21:26.692814 9146 heartbeater.cc:347] Connected to a master server at 127.4.59.62:39195
I20260620 09:21:26.692917 9146 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:26.693065 9146 heartbeater.cc:511] Master 127.4.59.62:39195 requested a full tablet report, sending...
I20260620 09:21:26.693569 8958 ts_manager.cc:194] Registered new tserver with Master: 7e038e0c75954b0b82051a4be44d7366 (127.4.59.1:46423)
I20260620 09:21:26.695736 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 9018
I20260620 09:21:26.695819 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:21:26.696022 4332 external_mini_cluster.cc:1428] Setting key 55ab4fa1e7899d9fb5f1238384b70f54
I20260620 09:21:26.696851 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:39195
--builtin_ntp_servers=127.4.59.20:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 09:21:26.775259 9149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:26.775450 9149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:26.775472 9149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:26.776640 9149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:26.776677 9149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:26.776703 9149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:21:26.777968 9149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:39195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:26.778146 9149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:26.778371 9149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:26.783424 9154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:26.783420 9157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:26.783622 9149 server_base.cc:1034] running on GCE node
W20260620 09:21:26.783684 9155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:26.783895 9149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:26.784124 9149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:26.785297 9149 hybrid_clock.cc:648] HybridClock initialized: now 1781947286785283 us; error 24 us; skew 500 ppm
I20260620 09:21:26.786566 9149 webserver.cc:460] Webserver started at http://127.4.59.2:45641/ using document root <none> and password file <none>
I20260620 09:21:26.786811 9149 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:26.786887 9149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:26.787029 9149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:26.788061 9149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "aba9afba82754b7e83b4d63193774e7d"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "85f56f9d460e46cee1d2312330d7ef37"
server_key_iv: "403de3c8060c747bbb219cc648da006e"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.788414 9149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "aba9afba82754b7e83b4d63193774e7d"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "85f56f9d460e46cee1d2312330d7ef37"
server_key_iv: "403de3c8060c747bbb219cc648da006e"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.789791 9149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:21:26.790532 9163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:26.790711 9149 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:26.790808 9149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "aba9afba82754b7e83b4d63193774e7d"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "85f56f9d460e46cee1d2312330d7ef37"
server_key_iv: "403de3c8060c747bbb219cc648da006e"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.790889 9149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:26.794174 9149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:26.794421 9149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:26.794564 9149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:26.794761 9149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:26.795048 9149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:26.795104 9149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:26.795146 9149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:26.795174 9149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:26.800688 9149 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:39609
I20260620 09:21:26.800750 9276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:39609 every 8 connection(s)
I20260620 09:21:26.801055 9149 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:21:26.801447 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 9149
I20260620 09:21:26.801542 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:21:26.801751 4332 external_mini_cluster.cc:1428] Setting key afdf45b76c246ce4cbf81b091afdc51d
I20260620 09:21:26.802670 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:39195
--builtin_ntp_servers=127.4.59.20:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260620 09:21:26.804080 9017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.801785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:60347 (local address 127.4.59.62:39195)
0620 09:21:26.801835 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.801837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.802126 (+ 289us) server_negotiation.cc:407] Connection header received
0620 09:21:26.802663 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.802664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.802671 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.802727 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:26.803844 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:26.803846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:26.803846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:21:26.803851 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:26.803867 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:21:26.803871 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:26.803917 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.803998 (+ 81us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.804011 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 09:21:26.804313 9279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.801780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:39195 (local address 127.4.59.2:60347)
0620 09:21:26.801989 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:26.802039 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:26.802563 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:26.802794 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:26.802798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:26.802807 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:26.803702 (+ 895us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:21:26.803703 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:26.803728 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:21:26.803754 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:26.803926 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:26.803930 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:26.803933 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:21:26.804006 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 09:21:26.804093 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
I20260620 09:21:26.804795 9277 heartbeater.cc:347] Connected to a master server at 127.4.59.62:39195
I20260620 09:21:26.804885 9277 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:26.805018 9277 heartbeater.cc:511] Master 127.4.59.62:39195 requested a full tablet report, sending...
I20260620 09:21:26.805388 8958 ts_manager.cc:194] Registered new tserver with Master: aba9afba82754b7e83b4d63193774e7d (127.4.59.2:39609)
W20260620 09:21:26.883239 9280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:26.883425 9280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:26.883447 9280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:26.884531 9280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:26.884564 9280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:26.884590 9280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:21:26.885869 9280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:39195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:26.886041 9280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:26.886272 9280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:26.891062 9288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:26.891093 9280 server_base.cc:1034] running on GCE node
W20260620 09:21:26.891062 9285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:26.891101 9286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:26.891530 9280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:26.891778 9280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:26.892917 9280 hybrid_clock.cc:648] HybridClock initialized: now 1781947286892902 us; error 33 us; skew 500 ppm
I20260620 09:21:26.894176 9280 webserver.cc:460] Webserver started at http://127.4.59.3:36639/ using document root <none> and password file <none>
I20260620 09:21:26.894476 9280 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:26.894524 9280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:26.894668 9280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:26.895668 9280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "e00738f766e946e0964ae3bafea960a5"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "ba7bb54e646b8db2f8fdd845c42decb2"
server_key_iv: "684293e34fc247674551b4dd4a6ecf99"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.896028 9280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "e00738f766e946e0964ae3bafea960a5"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "ba7bb54e646b8db2f8fdd845c42decb2"
server_key_iv: "684293e34fc247674551b4dd4a6ecf99"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.897356 9280 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:21:26.898149 9294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:26.898346 9280 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:26.898433 9280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "e00738f766e946e0964ae3bafea960a5"
format_stamp: "Formatted at 2026-06-20 09:21:26 on dist-test-slave-414w"
server_key: "ba7bb54e646b8db2f8fdd845c42decb2"
server_key_iv: "684293e34fc247674551b4dd4a6ecf99"
server_key_version: "encryptionkey@0"
I20260620 09:21:26.898515 9280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:26.901731 9280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:26.901983 9280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:26.902112 9280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:26.902329 9280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:26.902607 9280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:26.902658 9280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:26.902695 9280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:26.902721 9280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:26.908320 9280 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:35633
I20260620 09:21:26.908360 9407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:35633 every 8 connection(s)
I20260620 09:21:26.908655 9280 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:21:26.911531 9017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.909343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:55953 (local address 127.4.59.62:39195)
0620 09:21:26.909385 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.909387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.909595 (+ 208us) server_negotiation.cc:407] Connection header received
0620 09:21:26.910061 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.910062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.910068 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.910119 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:26.911291 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:26.911292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:26.911292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:21:26.911297 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:26.911310 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:21:26.911314 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:26.911371 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.911448 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.911463 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:21:26.911710 9410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.909329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:39195 (local address 127.4.59.3:55953)
0620 09:21:26.909487 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:26.909517 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:26.909961 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:26.910192 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:26.910196 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:26.910204 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:26.911129 (+ 925us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:21:26.911130 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:26.911154 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:21:26.911180 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:26.911370 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:26.911374 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:26.911377 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:21:26.911449 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 09:21:26.911505 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":65,"threads_started":1}
I20260620 09:21:26.912135 9408 heartbeater.cc:347] Connected to a master server at 127.4.59.62:39195
I20260620 09:21:26.912199 9408 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:26.912358 9408 heartbeater.cc:511] Master 127.4.59.62:39195 requested a full tablet report, sending...
I20260620 09:21:26.912685 8958 ts_manager.cc:194] Registered new tserver with Master: e00738f766e946e0964ae3bafea960a5 (127.4.59.3:35633)
I20260620 09:21:26.917533 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 9280
I20260620 09:21:26.917654 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:21:26.917822 4332 external_mini_cluster.cc:1428] Setting key 90519f644e41a798d2d7f26fee07c698
I20260620 09:21:26.918591 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:21:26.920877 9017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.919748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.4.59.62:39195)
0620 09:21:26.919824 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.919826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.919932 (+ 106us) server_negotiation.cc:407] Connection header received
0620 09:21:26.920009 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.920010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.920016 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.920085 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:26.920658 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:26.920659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:26.920660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:21:26.920664 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:26.920678 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:21:26.920682 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:26.920737 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.920793 (+ 56us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.920808 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 09:21:26.922349 8958 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51358:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:21:26.923812 8958 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:21:26.927318 9417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.925887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:46423 (local address 127.0.0.1:48390)
0620 09:21:26.926029 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:26.926085 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:26.926134 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:26.926473 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:26.926474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:26.926477 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:26.926899 (+ 422us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:21:26.926900 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:26.926912 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:21:26.926928 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:26.927159 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:26.927164 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:26.927167 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:21:26.927237 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 09:21:26.927252 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
I20260620 09:21:26.927337 9420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.926165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48390 (local address 127.4.59.1:46423)
0620 09:21:26.926310 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.926313 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.926324 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:21:26.926350 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.926353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.926363 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.926466 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:26.927019 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:26.927025 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:26.927025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:21:26.927029 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:26.927066 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:21:26.927092 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:26.927165 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.927244 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.927260 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260620 09:21:26.927801 9416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.925907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:39609 (local address 127.0.0.1:54902)
0620 09:21:26.926049 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:26.926079 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:26.926129 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:26.926426 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:26.926431 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:26.926437 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:26.927038 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:21:26.927038 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:26.927045 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:21:26.927057 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:26.927659 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:26.927660 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:26.927661 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 09:21:26.927720 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 09:21:26.927732 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":67,"threads_started":1}
I20260620 09:21:26.927841 9419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.926127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54902 (local address 127.4.59.2:39609)
0620 09:21:26.926271 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.926276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.926287 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:21:26.926314 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.926316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.926327 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.926957 (+ 630us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:26.927289 (+ 332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:26.927292 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:26.927292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:21:26.927295 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:26.927316 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:21:26.927324 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:26.927667 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.927748 (+ 81us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.927765 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260620 09:21:26.928018 9080 tablet_service.cc:1467] Processing CreateTablet for tablet 75cd94362aac42f9989bb39a4bc66237 (DEFAULT_TABLE table=test-table [id=2288612241ce4cf3b39ba56c5e1778d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:26.928400 9080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75cd94362aac42f9989bb39a4bc66237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:26.928768 9211 tablet_service.cc:1467] Processing CreateTablet for tablet 75cd94362aac42f9989bb39a4bc66237 (DEFAULT_TABLE table=test-table [id=2288612241ce4cf3b39ba56c5e1778d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:26.928900 9418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.925950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:35633 (local address 127.0.0.1:53386)
0620 09:21:26.926315 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:26.926324 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:26.926365 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:26.928127 (+ 1762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:26.928128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:26.928130 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:26.928541 (+ 411us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:21:26.928541 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:26.928547 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:21:26.928559 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:26.928784 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:26.928785 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:26.928786 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 09:21:26.928830 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 09:21:26.928843 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":90,"threads_started":1}
I20260620 09:21:26.929106 9211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75cd94362aac42f9989bb39a4bc66237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:26.929124 9421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:26.926329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53386 (local address 127.4.59.3:35633)
0620 09:21:26.927742 (+ 1413us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:26.927745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:26.927763 (+ 18us) server_negotiation.cc:407] Connection header received
0620 09:21:26.927785 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:26.927787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:26.927794 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:26.928123 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:26.928639 (+ 516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:26.928644 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:26.928645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:21:26.928649 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:26.928678 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:21:26.928697 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:26.928747 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:26.929026 (+ 279us) server_negotiation.cc:299] Negotiation successful
0620 09:21:26.929042 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1378,"thread_start_us":72,"threads_started":1}
I20260620 09:21:26.929670 9342 tablet_service.cc:1467] Processing CreateTablet for tablet 75cd94362aac42f9989bb39a4bc66237 (DEFAULT_TABLE table=test-table [id=2288612241ce4cf3b39ba56c5e1778d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:26.929968 9342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75cd94362aac42f9989bb39a4bc66237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:26.931337 9422 tablet_bootstrap.cc:492] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366: Bootstrap starting.
I20260620 09:21:26.932029 9422 tablet_bootstrap.cc:654] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:26.932343 9423 tablet_bootstrap.cc:492] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d: Bootstrap starting.
I20260620 09:21:26.932382 9422 log.cc:824] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:26.932955 9423 tablet_bootstrap.cc:654] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:26.933085 9422 tablet_bootstrap.cc:492] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366: No bootstrap required, opened a new log
I20260620 09:21:26.933183 9422 ts_tablet_manager.cc:1397] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:21:26.933293 9423 log.cc:824] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:26.933300 9425 tablet_bootstrap.cc:492] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5: Bootstrap starting.
I20260620 09:21:26.933930 9423 tablet_bootstrap.cc:492] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d: No bootstrap required, opened a new log
I20260620 09:21:26.933979 9423 ts_tablet_manager.cc:1397] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:21:26.934063 9425 tablet_bootstrap.cc:654] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:26.934404 9425 log.cc:824] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:26.935209 9425 tablet_bootstrap.cc:492] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5: No bootstrap required, opened a new log
I20260620 09:21:26.935328 9425 ts_tablet_manager.cc:1397] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:21:26.935808 9423 raft_consensus.cc:348] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e038e0c75954b0b82051a4be44d7366" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46423 } } peers { permanent_uuid: "e00738f766e946e0964ae3bafea960a5" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35633 } } peers { permanent_uuid: "aba9afba82754b7e83b4d63193774e7d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 39609 } }
I20260620 09:21:26.935926 9423 raft_consensus.cc:374] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:26.935946 9423 raft_consensus.cc:729] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aba9afba82754b7e83b4d63193774e7d, State: Initialized, Role: FOLLOWER
I20260620 09:21:26.936059 9423 consensus_queue.cc:260] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e038e0c75954b0b82051a4be44d7366" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46423 } } peers { permanent_uuid: "e00738f766e946e0964ae3bafea960a5" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35633 } } peers { permanent_uuid: "aba9afba82754b7e83b4d63193774e7d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 39609 } }
I20260620 09:21:26.936086 9422 raft_consensus.cc:348] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e038e0c75954b0b82051a4be44d7366" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46423 } } peers { permanent_uuid: "e00738f766e946e0964ae3bafea960a5" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35633 } } peers { permanent_uuid: "aba9afba82754b7e83b4d63193774e7d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 39609 } }
I20260620 09:21:26.936239 9422 raft_consensus.cc:374] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:26.936280 9422 raft_consensus.cc:729] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e038e0c75954b0b82051a4be44d7366, State: Initialized, Role: FOLLOWER
I20260620 09:21:26.936309 9277 heartbeater.cc:503] Master 127.4.59.62:39195 was elected leader, sending a full tablet report...
I20260620 09:21:26.936398 9422 consensus_queue.cc:260] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e038e0c75954b0b82051a4be44d7366" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46423 } } peers { permanent_uuid: "e00738f766e946e0964ae3bafea960a5" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35633 } } peers { permanent_uuid: "aba9afba82754b7e83b4d63193774e7d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 39609 } }
I20260620 09:21:26.936308 9423 ts_tablet_manager.cc:1428] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 09:21:26.937168 9422 ts_tablet_manager.cc:1428] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260620 09:21:26.937330 9146 heartbeater.cc:503] Master 127.4.59.62:39195 was elected leader, sending a full tablet report...
W20260620 09:21:26.938349 9147 tablet.cc:2369] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:21:26.938540 9425 raft_consensus.cc:348] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e038e0c75954b0b82051a4be44d7366" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46423 } } peers { permanent_uuid: "e00738f766e946e0964ae3bafea960a5" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35633 } } peers { permanent_uuid: "aba9afba82754b7e83b4d63193774e7d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 39609 } }
I20260620 09:21:26.938679 9425 raft_consensus.cc:374] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:26.938723 9425 raft_consensus.cc:729] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e00738f766e946e0964ae3bafea960a5, State: Initialized, Role: FOLLOWER
I20260620 09:21:26.938824 9425 consensus_queue.cc:260] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e038e0c75954b0b82051a4be44d7366" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46423 } } peers { permanent_uuid: "e00738f766e946e0964ae3bafea960a5" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35633 } } peers { permanent_uuid: "aba9afba82754b7e83b4d63193774e7d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 39609 } }
I20260620 09:21:26.939064 9425 ts_tablet_manager.cc:1428] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260620 09:21:26.939117 9408 heartbeater.cc:503] Master 127.4.59.62:39195 was elected leader, sending a full tablet report...
W20260620 09:21:27.051695 9278 tablet.cc:2369] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 09:21:27.159696 9409 tablet.cc:2369] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:21:27.197356 9429 raft_consensus.cc:482] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:21:27.197474 9429 raft_consensus.cc:504] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e038e0c75954b0b82051a4be44d7366" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46423 } } peers { permanent_uuid: "e00738f766e946e0964ae3bafea960a5" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35633 } } peers { permanent_uuid: "aba9afba82754b7e83b4d63193774e7d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 39609 } }
I20260620 09:21:27.197811 9429 leader_election.cc:288] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e00738f766e946e0964ae3bafea960a5 (127.4.59.3:35633), aba9afba82754b7e83b4d63193774e7d (127.4.59.2:39609)
I20260620 09:21:27.199319 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:27.197965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:39609 (local address 127.4.59.1:42747)
0620 09:21:27.198085 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:27.198096 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:27.198155 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:27.198312 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:27.198313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:27.198316 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:27.198853 (+ 537us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:21:27.198853 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:27.198861 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:21:27.198875 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:27.199136 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:27.199138 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:27.199138 (+ 0us) client_negotiation.cc:769] Sending connection context
0620 09:21:27.199197 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 09:21:27.199211 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260620 09:21:27.199342 9419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:27.198039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:42747 (local address 127.4.59.2:39609)
0620 09:21:27.198112 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:27.198114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:27.198146 (+ 32us) server_negotiation.cc:407] Connection header received
0620 09:21:27.198233 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:27.198234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:27.198240 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:27.198298 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:27.198975 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:27.198977 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:27.198977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:21:27.198985 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:27.199020 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:21:27.199026 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:27.199126 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:27.199203 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 09:21:27.199235 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 09:21:27.199414 9421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:27.198068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:37277 (local address 127.4.59.3:35633)
0620 09:21:27.198126 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:27.198128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:27.198199 (+ 71us) server_negotiation.cc:407] Connection header received
0620 09:21:27.198297 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:27.198298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:27.198303 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:27.198377 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:27.199150 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:27.199151 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:27.199151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:21:27.199156 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:27.199173 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:21:27.199180 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:27.199234 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:27.199324 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 09:21:27.199340 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 09:21:27.199429 9432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:27.197966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:35633 (local address 127.4.59.1:37277)
0620 09:21:27.198116 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:27.198127 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:27.198215 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:27.198384 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:27.198385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:27.198390 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:27.199049 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:21:27.199049 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:27.199056 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:21:27.199069 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:27.199245 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:27.199245 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:27.199246 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 09:21:27.199322 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 09:21:27.199334 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":13,"thread_start_us":58,"threads_started":1}
I20260620 09:21:27.199759 9362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75cd94362aac42f9989bb39a4bc66237" candidate_uuid: "7e038e0c75954b0b82051a4be44d7366" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e00738f766e946e0964ae3bafea960a5" is_pre_election: true
I20260620 09:21:27.199856 9231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75cd94362aac42f9989bb39a4bc66237" candidate_uuid: "7e038e0c75954b0b82051a4be44d7366" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aba9afba82754b7e83b4d63193774e7d" is_pre_election: true
I20260620 09:21:27.199913 9362 raft_consensus.cc:2454] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e038e0c75954b0b82051a4be44d7366 in term 0.
I20260620 09:21:27.200011 9231 raft_consensus.cc:2454] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e038e0c75954b0b82051a4be44d7366 in term 0.
I20260620 09:21:27.200181 9033 leader_election.cc:302] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e038e0c75954b0b82051a4be44d7366, e00738f766e946e0964ae3bafea960a5; no voters:
I20260620 09:21:27.200285 9429 raft_consensus.cc:2781] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:21:27.200344 9429 raft_consensus.cc:482] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:21:27.200367 9429 raft_consensus.cc:3037] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:27.201076 9429 raft_consensus.cc:504] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e038e0c75954b0b82051a4be44d7366" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46423 } } peers { permanent_uuid: "e00738f766e946e0964ae3bafea960a5" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35633 } } peers { permanent_uuid: "aba9afba82754b7e83b4d63193774e7d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 39609 } }
I20260620 09:21:27.201231 9429 leader_election.cc:288] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [CANDIDATE]: Term 1 election: Requested vote from peers e00738f766e946e0964ae3bafea960a5 (127.4.59.3:35633), aba9afba82754b7e83b4d63193774e7d (127.4.59.2:39609)
I20260620 09:21:27.201395 9231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75cd94362aac42f9989bb39a4bc66237" candidate_uuid: "7e038e0c75954b0b82051a4be44d7366" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aba9afba82754b7e83b4d63193774e7d"
I20260620 09:21:27.201373 9362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75cd94362aac42f9989bb39a4bc66237" candidate_uuid: "7e038e0c75954b0b82051a4be44d7366" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e00738f766e946e0964ae3bafea960a5"
I20260620 09:21:27.201469 9231 raft_consensus.cc:3037] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:27.201467 9362 raft_consensus.cc:3037] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:27.202174 9231 raft_consensus.cc:2454] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e038e0c75954b0b82051a4be44d7366 in term 1.
I20260620 09:21:27.202240 9362 raft_consensus.cc:2454] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e038e0c75954b0b82051a4be44d7366 in term 1.
I20260620 09:21:27.202375 9034 leader_election.cc:302] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e038e0c75954b0b82051a4be44d7366, aba9afba82754b7e83b4d63193774e7d; no voters:
I20260620 09:21:27.202491 9429 raft_consensus.cc:2781] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:27.202643 9429 raft_consensus.cc:686] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [term 1 LEADER]: Becoming Leader. State: Replica: 7e038e0c75954b0b82051a4be44d7366, State: Running, Role: LEADER
I20260620 09:21:27.202731 9429 consensus_queue.cc:237] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e038e0c75954b0b82051a4be44d7366" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46423 } } peers { permanent_uuid: "e00738f766e946e0964ae3bafea960a5" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35633 } } peers { permanent_uuid: "aba9afba82754b7e83b4d63193774e7d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 39609 } }
I20260620 09:21:27.203529 8958 catalog_manager.cc:5495] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e038e0c75954b0b82051a4be44d7366 (127.4.59.1). New cstate: current_term: 1 leader_uuid: "7e038e0c75954b0b82051a4be44d7366" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e038e0c75954b0b82051a4be44d7366" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e00738f766e946e0964ae3bafea960a5" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aba9afba82754b7e83b4d63193774e7d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 39609 } health_report { overall_health: UNKNOWN } } }
I20260620 09:21:27.208667 9420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:27.207797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48402 (local address 127.4.59.1:46423)
0620 09:21:27.207852 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:27.207854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:27.207862 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:21:27.207929 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:27.207930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:27.207940 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:27.207996 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:27.208462 (+ 466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:27.208464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:27.208464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:21:27.208468 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:27.208482 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:21:27.208486 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:27.208539 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:27.208592 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 09:21:27.208605 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 09:21:27.210186 9231 raft_consensus.cc:1261] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d [term 1 FOLLOWER]: Refusing update from remote peer 7e038e0c75954b0b82051a4be44d7366: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:27.210186 9362 raft_consensus.cc:1261] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5 [term 1 FOLLOWER]: Refusing update from remote peer 7e038e0c75954b0b82051a4be44d7366: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:27.210501 9433 consensus_queue.cc:1035] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aba9afba82754b7e83b4d63193774e7d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 39609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:27.210629 9429 consensus_queue.cc:1035] T 75cd94362aac42f9989bb39a4bc66237 P 7e038e0c75954b0b82051a4be44d7366 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e00738f766e946e0964ae3bafea960a5" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:27.214965 8958 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:27.215051 8958 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:27.215720 8958 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d45aa95057e54e93a6681dbacc4721d5: Sending DeleteTablet for 3 replicas of tablet 75cd94362aac42f9989bb39a4bc66237
I20260620 09:21:27.216018 9080 tablet_service.cc:1514] Processing DeleteTablet for tablet 75cd94362aac42f9989bb39a4bc66237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:27 UTC) from {username='slave'} at 127.0.0.1:48390
I20260620 09:21:27.216256 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 9018
I20260620 09:21:27.216241 9342 tablet_service.cc:1514] Processing DeleteTablet for tablet 75cd94362aac42f9989bb39a4bc66237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:27 UTC) from {username='slave'} at 127.0.0.1:53386
I20260620 09:21:27.217088 9211 tablet_service.cc:1514] Processing DeleteTablet for tablet 75cd94362aac42f9989bb39a4bc66237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:27 UTC) from {username='slave'} at 127.0.0.1:54902
I20260620 09:21:27.217123 9446 tablet_replica.cc:330] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5: stopping tablet replica
I20260620 09:21:27.217254 9446 raft_consensus.cc:2229] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:27.217300 9447 tablet_replica.cc:330] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d: stopping tablet replica
I20260620 09:21:27.217346 9446 raft_consensus.cc:2258] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:27.217371 9447 raft_consensus.cc:2229] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:27.217451 9447 raft_consensus.cc:2258] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:27.217741 9446 ts_tablet_manager.cc:1905] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:27.218045 9447 ts_tablet_manager.cc:1905] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:27.219297 9446 ts_tablet_manager.cc:1918] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:21:27.219370 9446 log.cc:1196] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/75cd94362aac42f9989bb39a4bc66237
I20260620 09:21:27.219594 9447 ts_tablet_manager.cc:1918] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:21:27.219630 9446 ts_tablet_manager.cc:1939] T 75cd94362aac42f9989bb39a4bc66237 P e00738f766e946e0964ae3bafea960a5: Deleting consensus metadata
I20260620 09:21:27.219662 9447 log.cc:1196] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/75cd94362aac42f9989bb39a4bc66237
I20260620 09:21:27.219904 9447 ts_tablet_manager.cc:1939] T 75cd94362aac42f9989bb39a4bc66237 P aba9afba82754b7e83b4d63193774e7d: Deleting consensus metadata
I20260620 09:21:27.220002 8942 catalog_manager.cc:4841] TS e00738f766e946e0964ae3bafea960a5 (127.4.59.3:35633): tablet 75cd94362aac42f9989bb39a4bc66237 (table test-table [id=2288612241ce4cf3b39ba56c5e1778d7]) successfully deleted
I20260620 09:21:27.220297 8943 catalog_manager.cc:4841] TS aba9afba82754b7e83b4d63193774e7d (127.4.59.2:39609): tablet 75cd94362aac42f9989bb39a4bc66237 (table test-table [id=2288612241ce4cf3b39ba56c5e1778d7]) successfully deleted
W20260620 09:21:27.222224 8944 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.59.1:46423 (error 108)
I20260620 09:21:27.222688 9418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:27.222566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:46423 (local address 127.0.0.1:48404)
0620 09:21:27.222617 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:27.222654 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:46423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 09:21:27.222684 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 9149
W20260620 09:21:27.223222 8944 catalog_manager.cc:4584] TS 7e038e0c75954b0b82051a4be44d7366 (127.4.59.1:46423): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 75cd94362aac42f9989bb39a4bc66237: Network error: Client connection negotiation failed: client connection to 127.4.59.1:46423: connect: Connection refused (error 111)
I20260620 09:21:27.227902 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 9280
I20260620 09:21:27.232010 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 8927
2026-06-20T09:21:27Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (825 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260620 09:21:27.250833 4332 mini_oidc.cc:152] Starting JWKS server
I20260620 09:21:27.251822 4332 webserver.cc:460] Webserver started at https://127.0.0.1:35005/ using document root <none> and password file <none>
I20260620 09:21:27.251893 4332 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 09:21:27.252087 4332 webserver.cc:460] Webserver started at http://127.0.0.1:35459/ using document root <none> and password file <none>
2026-06-20T09:21:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:21:27Z Disabled control of system clock
I20260620 09:21:27.259150 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:44223
--webserver_interface=127.4.59.62
--webserver_port=0
--builtin_ntp_servers=127.4.59.20:42477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:44223
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35005/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/testchainca.cert with env {}
W20260620 09:21:27.335659 9454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:27.335847 9454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:27.335870 9454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:27.337018 9454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:27.337059 9454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:27.337071 9454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:27.337083 9454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:27.337131 9454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 09:21:27.337177 9454 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 09:21:27.337196 9454 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35005/jwks.json
I20260620 09:21:27.338662 9454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:44223
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:44223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35005/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:27.338909 9454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:27.339118 9454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 09:21:27.343621 9454 server_base.cc:1034] running on GCE node
W20260620 09:21:27.343675 9459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:27.343719 9460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:27.343619 9462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:27.344067 9454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:27.344347 9454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:27.345520 9454 hybrid_clock.cc:648] HybridClock initialized: now 1781947287345503 us; error 45 us; skew 500 ppm
I20260620 09:21:27.346698 9454 webserver.cc:460] Webserver started at http://127.4.59.62:42995/ using document root <none> and password file <none>
I20260620 09:21:27.346931 9454 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:27.347000 9454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:27.347148 9454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:27.348094 9454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "9e20393fbaa04c978dbb661a60b38350"
format_stamp: "Formatted at 2026-06-20 09:21:27 on dist-test-slave-414w"
server_key: "e46ae0f63f9dc8e7cf1d3c9a0adb593e"
server_key_iv: "c48aed83d40dcfcd3f77699cc7af263b"
server_key_version: "encryptionkey@0"
I20260620 09:21:27.348439 9454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "9e20393fbaa04c978dbb661a60b38350"
format_stamp: "Formatted at 2026-06-20 09:21:27 on dist-test-slave-414w"
server_key: "e46ae0f63f9dc8e7cf1d3c9a0adb593e"
server_key_iv: "c48aed83d40dcfcd3f77699cc7af263b"
server_key_version: "encryptionkey@0"
I20260620 09:21:27.349720 9454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:21:27.350453 9468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:27.350615 9454 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:27.350709 9454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "9e20393fbaa04c978dbb661a60b38350"
format_stamp: "Formatted at 2026-06-20 09:21:27 on dist-test-slave-414w"
server_key: "e46ae0f63f9dc8e7cf1d3c9a0adb593e"
server_key_iv: "c48aed83d40dcfcd3f77699cc7af263b"
server_key_version: "encryptionkey@0"
I20260620 09:21:27.350782 9454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:27.377771 9454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:27.378051 9454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:27.378196 9454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:27.381947 9454 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:44223
I20260620 09:21:27.381979 9520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:44223 every 8 connection(s)
I20260620 09:21:27.382296 9454 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:21:27.382864 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 9454
I20260620 09:21:27.382975 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:21:27.383144 4332 external_mini_cluster.cc:1428] Setting key ce40cadc15b7e2cde53716b020f17314
I20260620 09:21:27.383389 9521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:27.387096 9521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350: Bootstrap starting.
I20260620 09:21:27.387856 9521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:27.388206 9521 log.cc:824] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:27.388985 9521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350: No bootstrap required, opened a new log
I20260620 09:21:27.389616 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:27.383638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58280 (local address 127.4.59.62:44223)
0620 09:21:27.383866 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:27.383875 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:27.383916 (+ 41us) server_negotiation.cc:407] Connection header received
0620 09:21:27.384632 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:27.384642 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:27.384731 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:27.384893 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:27.385065 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.386558 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.387387 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.388080 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.388219 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:27.388859 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:27.388863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:27.388864 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:21:27.389075 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:27.389123 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:21:27.389165 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:27.389244 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:27.389302 (+ 58us) server_negotiation.cc:299] Negotiation successful
0620 09:21:27.389380 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260620 09:21:27.391651 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:21:27.391701 9521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e20393fbaa04c978dbb661a60b38350" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 44223 } }
I20260620 09:21:27.391884 9521 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:27.391932 9521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e20393fbaa04c978dbb661a60b38350, State: Initialized, Role: FOLLOWER
I20260620 09:21:27.392055 9521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e20393fbaa04c978dbb661a60b38350" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 44223 } }
I20260620 09:21:27.392138 9521 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:21:27.392184 9521 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:21:27.392235 9521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:27.392781 9521 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e20393fbaa04c978dbb661a60b38350" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 44223 } }
I20260620 09:21:27.392899 9521 leader_election.cc:302] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e20393fbaa04c978dbb661a60b38350; no voters:
I20260620 09:21:27.393064 9521 leader_election.cc:288] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:21:27.393169 9526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:27.393297 9526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [term 1 LEADER]: Becoming Leader. State: Replica: 9e20393fbaa04c978dbb661a60b38350, State: Running, Role: LEADER
I20260620 09:21:27.393373 9521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:27.393381 9526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e20393fbaa04c978dbb661a60b38350" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 44223 } }
I20260620 09:21:27.393739 9526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e20393fbaa04c978dbb661a60b38350. Latest consensus state: current_term: 1 leader_uuid: "9e20393fbaa04c978dbb661a60b38350" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e20393fbaa04c978dbb661a60b38350" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 44223 } } }
I20260620 09:21:27.393749 9527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e20393fbaa04c978dbb661a60b38350" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e20393fbaa04c978dbb661a60b38350" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 44223 } } }
I20260620 09:21:27.393829 9526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:27.393882 9527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:27.394059 9532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:21:27.394731 9532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:21:27.396464 9532 catalog_manager.cc:1347] Generated new cluster ID: f348ff95738e498cb485f5f62dc4c58c
I20260620 09:21:27.396512 9532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:21:27.404728 9532 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:21:27.405258 9532 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:21:27.412637 9532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e20393fbaa04c978dbb661a60b38350: Generated new TSK 0
I20260620 09:21:27.412819 9532 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 09:21:27.437278 9551 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260620 09:21:27.437367 9551 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260620 09:21:27.437891 9550 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:21:27.421752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:44223 (local address 127.0.0.1:58296)
0620 09:21:27.421941 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:27.421951 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:27.422004 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:27.422171 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:27.422171 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:27.422173 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 09:21:27.422223 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:27.422224 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.423192 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.423193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:27.423779 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:27.423780 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.424260 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.424264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:27.424462 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:27.424483 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 09:21:27.437511 (+ 13028us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35005/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 09:21:27.437597 (+ 86us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.59.62:44223: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35005/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260620 09:21:27.438158 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:27.421832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58296 (local address 127.4.59.62:44223)
0620 09:21:27.421890 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:27.421892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:27.422006 (+ 114us) server_negotiation.cc:407] Connection header received
0620 09:21:27.422081 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:27.422082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:27.422095 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:27.422159 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0620 09:21:27.422281 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.423104 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.423859 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.424164 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.424238 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:27.424607 (+ 369us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 09:21:27.437383 (+ 12776us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35005/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 09:21:27.438088 (+ 705us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58296: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35005/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":34}
W20260620 09:21:27.438201 9524 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58296: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35005/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.4.59.62:44223: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35005/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260620 09:21:27.438346 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 9454
2026-06-20T09:21:27Z chronyd exiting
I20260620 09:21:27.473781 4332 test_util.cc:175] -----------------------------------------------
I20260620 09:21:27.473876 4332 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781947262065055-4332-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (223 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260620 09:21:27.474611 4332 mini_oidc.cc:152] Starting JWKS server
I20260620 09:21:27.475282 4332 webserver.cc:460] Webserver started at https://127.0.0.1:46247/ using document root <none> and password file <none>
I20260620 09:21:27.475348 4332 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 09:21:27.475507 4332 webserver.cc:460] Webserver started at http://127.0.0.1:33093/ using document root <none> and password file <none>
2026-06-20T09:21:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:21:27Z Disabled control of system clock
I20260620 09:21:27.482887 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:32863
--webserver_interface=127.4.59.62
--webserver_port=0
--builtin_ntp_servers=127.4.59.20:39101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:32863
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46247/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/testchainca.cert with env {}
W20260620 09:21:27.560855 9557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:27.561048 9557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:27.561069 9557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:27.562222 9557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:27.562263 9557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:27.562278 9557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:27.562290 9557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:27.562305 9557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 09:21:27.562363 9557 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 09:21:27.562384 9557 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46247/jwks.json
I20260620 09:21:27.563866 9557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:39101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:32863
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:32863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46247/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:27.564100 9557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:27.564302 9557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 09:21:27.569008 9557 server_base.cc:1034] running on GCE node
W20260620 09:21:27.568959 9562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:27.568958 9563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:27.568957 9565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:27.569432 9557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:27.569736 9557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:27.570899 9557 hybrid_clock.cc:648] HybridClock initialized: now 1781947287570896 us; error 41 us; skew 500 ppm
I20260620 09:21:27.572045 9557 webserver.cc:460] Webserver started at http://127.4.59.62:46571/ using document root <none> and password file <none>
I20260620 09:21:27.572273 9557 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:27.572346 9557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:27.572494 9557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:27.573566 9557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "08c0c7744c89422b8aa16f4ddeab97d7"
format_stamp: "Formatted at 2026-06-20 09:21:27 on dist-test-slave-414w"
server_key: "0a38d569d51c6e74acf909201fe77d7e"
server_key_iv: "47b1a6ab01bd74dfa0d8bdd877a70022"
server_key_version: "encryptionkey@0"
I20260620 09:21:27.573909 9557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "08c0c7744c89422b8aa16f4ddeab97d7"
format_stamp: "Formatted at 2026-06-20 09:21:27 on dist-test-slave-414w"
server_key: "0a38d569d51c6e74acf909201fe77d7e"
server_key_iv: "47b1a6ab01bd74dfa0d8bdd877a70022"
server_key_version: "encryptionkey@0"
I20260620 09:21:27.575125 9557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:21:27.575814 9571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:27.575963 9557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:27.576058 9557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "08c0c7744c89422b8aa16f4ddeab97d7"
format_stamp: "Formatted at 2026-06-20 09:21:27 on dist-test-slave-414w"
server_key: "0a38d569d51c6e74acf909201fe77d7e"
server_key_iv: "47b1a6ab01bd74dfa0d8bdd877a70022"
server_key_version: "encryptionkey@0"
I20260620 09:21:27.576151 9557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:27.612448 9557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:27.612776 9557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:27.612936 9557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:27.616732 9557 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:32863
I20260620 09:21:27.616801 9623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:32863 every 8 connection(s)
I20260620 09:21:27.617157 9557 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:21:27.618086 9624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:27.620260 9624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7: Bootstrap starting.
I20260620 09:21:27.620815 9624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:27.621098 9624 log.cc:824] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:27.621781 9624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7: No bootstrap required, opened a new log
I20260620 09:21:27.623273 9624 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c0c7744c89422b8aa16f4ddeab97d7" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 32863 } }
I20260620 09:21:27.623414 9624 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:27.623437 9624 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c0c7744c89422b8aa16f4ddeab97d7, State: Initialized, Role: FOLLOWER
I20260620 09:21:27.623597 9624 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c0c7744c89422b8aa16f4ddeab97d7" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 32863 } }
I20260620 09:21:27.623657 9624 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:21:27.623677 9624 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:21:27.623745 9624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:27.624253 9624 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c0c7744c89422b8aa16f4ddeab97d7" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 32863 } }
I20260620 09:21:27.624334 9624 leader_election.cc:302] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08c0c7744c89422b8aa16f4ddeab97d7; no voters:
I20260620 09:21:27.624531 9624 leader_election.cc:288] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:21:27.624569 9627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:27.624735 9627 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [term 1 LEADER]: Becoming Leader. State: Replica: 08c0c7744c89422b8aa16f4ddeab97d7, State: Running, Role: LEADER
I20260620 09:21:27.624785 9624 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:27.624857 9627 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c0c7744c89422b8aa16f4ddeab97d7" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 32863 } }
I20260620 09:21:27.625244 9628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08c0c7744c89422b8aa16f4ddeab97d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c0c7744c89422b8aa16f4ddeab97d7" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 32863 } } }
I20260620 09:21:27.625344 9629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08c0c7744c89422b8aa16f4ddeab97d7. Latest consensus state: current_term: 1 leader_uuid: "08c0c7744c89422b8aa16f4ddeab97d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c0c7744c89422b8aa16f4ddeab97d7" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 32863 } } }
I20260620 09:21:27.625393 9628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:27.625428 9629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:27.625602 9636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:21:27.626113 9636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:21:27.626802 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 9557
I20260620 09:21:27.626924 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:21:27.627115 4332 external_mini_cluster.cc:1428] Setting key 2012ff43ff36445e86d3230a35cd5754
I20260620 09:21:27.627854 9636 catalog_manager.cc:1347] Generated new cluster ID: 656f83bf57b54ec4b82fcae03652d467
I20260620 09:21:27.627908 9636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:21:27.633270 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:27.627536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38140 (local address 127.4.59.62:32863)
0620 09:21:27.627828 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:27.627836 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:27.627863 (+ 27us) server_negotiation.cc:407] Connection header received
0620 09:21:27.627977 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:27.627984 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:27.628049 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:27.628397 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:27.628485 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.629863 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.630998 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.631622 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.632058 (+ 436us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:27.632442 (+ 384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:27.632446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:27.632446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:21:27.632638 (+ 192us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:27.632677 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:21:27.632691 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:27.632934 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:27.632966 (+ 32us) server_negotiation.cc:299] Negotiation successful
0620 09:21:27.633042 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":193,"threads_started":1}
I20260620 09:21:27.634281 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:21:27.656920 9636 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:21:27.657488 9636 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:21:27.661528 9636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08c0c7744c89422b8aa16f4ddeab97d7: Generated new TSK 0
I20260620 09:21:27.661697 9636 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 09:21:27.681499 9654 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260620 09:21:27.681567 9654 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 09:21:27.681937 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:27.667661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38152 (local address 127.4.59.62:32863)
0620 09:21:27.667700 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:27.667702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:27.667779 (+ 77us) server_negotiation.cc:407] Connection header received
0620 09:21:27.667824 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:27.667825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:27.667834 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:27.667878 (+ 44us) server_negotiation.cc:226] Negotiated authn=JWT
0620 09:21:27.668039 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.668544 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.669689 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.669875 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.669928 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:27.670117 (+ 189us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 09:21:27.681701 (+ 11584us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 09:21:27.681862 (+ 161us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38152: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 09:21:27.681989 9647 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38152: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260620 09:21:27.682020 9653 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:21:27.667562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:32863 (local address 127.0.0.1:38152)
0620 09:21:27.667700 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:27.667711 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:27.667772 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:27.667888 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:27.667889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:27.667891 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 09:21:27.667948 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:27.667950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.668624 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.668625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:27.669600 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:27.669603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.669950 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.669953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:27.670011 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:27.670016 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 09:21:27.681823 (+ 11807us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 09:21:27.681919 (+ 96us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.59.62:32863: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":94,"threads_started":1}
I20260620 09:21:27.682217 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 9557
2026-06-20T09:21:27Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (244 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260620 09:21:27.719533 4332 mini_oidc.cc:152] Starting JWKS server
I20260620 09:21:27.720221 4332 webserver.cc:460] Webserver started at https://127.0.0.1:39451/ using document root <none> and password file <none>
I20260620 09:21:27.720289 4332 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 09:21:27.720463 4332 webserver.cc:460] Webserver started at http://127.0.0.1:36191/ using document root <none> and password file <none>
2026-06-20T09:21:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:21:27Z Disabled control of system clock
I20260620 09:21:27.727746 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:40577
--webserver_interface=127.4.59.62
--webserver_port=0
--builtin_ntp_servers=127.4.59.20:33215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:40577
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39451/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 09:21:27.808826 9660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:27.809031 9660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:27.809051 9660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:27.810228 9660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:27.810272 9660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:27.810285 9660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:27.810298 9660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:27.810310 9660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 09:21:27.810322 9660 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 09:21:27.810334 9660 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39451/jwks.json
I20260620 09:21:27.811601 9660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:33215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:40577
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:40577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39451/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:27.811786 9660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:27.812007 9660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:27.816596 9665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:27.816601 9668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:27.816601 9666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:27.816782 9660 server_base.cc:1034] running on GCE node
I20260620 09:21:27.817211 9660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:27.817529 9660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:27.818674 9660 hybrid_clock.cc:648] HybridClock initialized: now 1781947287818647 us; error 38 us; skew 500 ppm
I20260620 09:21:27.819919 9660 webserver.cc:460] Webserver started at http://127.4.59.62:34783/ using document root <none> and password file <none>
I20260620 09:21:27.820227 9660 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:27.820310 9660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:27.820483 9660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:27.821761 9660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "311ce16eb9d040048898ed1ac500d097"
format_stamp: "Formatted at 2026-06-20 09:21:27 on dist-test-slave-414w"
server_key: "a1a0c808b16f5fa242a2a0cfd211cc18"
server_key_iv: "310196c30ee962d459e587a36b2cc204"
server_key_version: "encryptionkey@0"
I20260620 09:21:27.822161 9660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "311ce16eb9d040048898ed1ac500d097"
format_stamp: "Formatted at 2026-06-20 09:21:27 on dist-test-slave-414w"
server_key: "a1a0c808b16f5fa242a2a0cfd211cc18"
server_key_iv: "310196c30ee962d459e587a36b2cc204"
server_key_version: "encryptionkey@0"
I20260620 09:21:27.824029 9660 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260620 09:21:27.824838 9674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:27.825017 9660 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:27.825124 9660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "311ce16eb9d040048898ed1ac500d097"
format_stamp: "Formatted at 2026-06-20 09:21:27 on dist-test-slave-414w"
server_key: "a1a0c808b16f5fa242a2a0cfd211cc18"
server_key_iv: "310196c30ee962d459e587a36b2cc204"
server_key_version: "encryptionkey@0"
I20260620 09:21:27.825259 9660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:27.834445 9660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:27.834743 9660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:27.834900 9660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:27.838605 9660 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:40577
I20260620 09:21:27.838650 9726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:40577 every 8 connection(s)
I20260620 09:21:27.838997 9660 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:21:27.839665 9727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:27.841591 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 9660
I20260620 09:21:27.841703 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:21:27.841892 4332 external_mini_cluster.cc:1428] Setting key 8b8ae2229b45758868888ae5f83be632
I20260620 09:21:27.842578 9727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097: Bootstrap starting.
I20260620 09:21:27.843266 9727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:27.843600 9727 log.cc:824] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:27.844345 9727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097: No bootstrap required, opened a new log
I20260620 09:21:27.846437 9727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311ce16eb9d040048898ed1ac500d097" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40577 } }
I20260620 09:21:27.846611 9727 raft_consensus.cc:374] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:27.846655 9727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 311ce16eb9d040048898ed1ac500d097, State: Initialized, Role: FOLLOWER
I20260620 09:21:27.846815 9727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311ce16eb9d040048898ed1ac500d097" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40577 } }
I20260620 09:21:27.846907 9727 raft_consensus.cc:388] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:21:27.846951 9727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:21:27.847031 9727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:27.847636 9727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311ce16eb9d040048898ed1ac500d097" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40577 } }
I20260620 09:21:27.847748 9727 leader_election.cc:302] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 311ce16eb9d040048898ed1ac500d097; no voters:
I20260620 09:21:27.847906 9727 leader_election.cc:288] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:21:27.847963 9732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:27.848153 9732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [term 1 LEADER]: Becoming Leader. State: Replica: 311ce16eb9d040048898ed1ac500d097, State: Running, Role: LEADER
I20260620 09:21:27.848284 9732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311ce16eb9d040048898ed1ac500d097" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40577 } }
I20260620 09:21:27.848366 9730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:27.842431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40254 (local address 127.4.59.62:40577)
0620 09:21:27.842639 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:27.842649 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:27.842682 (+ 33us) server_negotiation.cc:407] Connection header received
0620 09:21:27.843296 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:27.843304 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:27.843387 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:27.843538 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:27.843724 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.845025 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.845950 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.846588 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.846709 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:27.847497 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:27.847501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:27.847502 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:21:27.847706 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:27.847751 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:21:27.847767 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:27.847886 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:27.848027 (+ 141us) server_negotiation.cc:299] Negotiation successful
0620 09:21:27.848129 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260620 09:21:27.848170 9727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:27.848678 9734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "311ce16eb9d040048898ed1ac500d097" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311ce16eb9d040048898ed1ac500d097" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40577 } } }
I20260620 09:21:27.848778 9734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:27.848994 9736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:21:27.849449 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:21:27.848665 9733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 311ce16eb9d040048898ed1ac500d097. Latest consensus state: current_term: 1 leader_uuid: "311ce16eb9d040048898ed1ac500d097" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311ce16eb9d040048898ed1ac500d097" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40577 } } }
I20260620 09:21:27.849506 9736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:21:27.849519 9733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:27.850973 9736 catalog_manager.cc:1347] Generated new cluster ID: 37cfd3cad17c494cb01ee5c3acbad472
I20260620 09:21:27.851025 9736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:21:27.868096 9736 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:21:27.868618 9736 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:21:27.872083 9736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 311ce16eb9d040048898ed1ac500d097: Generated new TSK 0
I20260620 09:21:27.872270 9736 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:21:27.883260 9730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:27.879536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40258 (local address 127.4.59.62:40577)
0620 09:21:27.879582 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:27.879585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:27.879685 (+ 100us) server_negotiation.cc:407] Connection header received
0620 09:21:27.879726 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:27.879727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:27.879739 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:27.879819 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0620 09:21:27.879924 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.880739 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.881564 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.881881 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.881946 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:27.882097 (+ 151us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 09:21:27.883048 (+ 951us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39451/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 09:21:27.883188 (+ 140us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40258: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39451/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":25}
W20260620 09:21:27.883314 9730 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40258: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39451/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260620 09:21:27.885475 9756 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:21:27.879458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40577 (local address 127.0.0.1:40258)
0620 09:21:27.879609 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:27.879619 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:27.879679 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:27.879825 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:27.879826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:27.879828 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 09:21:27.879875 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:27.879877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.880828 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.880829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:27.881489 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:27.881492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:27.881958 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:27.881965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:27.882004 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:27.882008 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 09:21:27.883402 (+ 1394us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39451/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 09:21:27.885372 (+ 1970us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.59.62:40577: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39451/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":1775,"thread_start_us":98,"threads_started":1}
I20260620 09:21:27.885725 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 9660
E20260620 09:21:27.894812 9757 webserver.cc:533] Webserver: sslize failed:
E20260620 09:21:27.894876 9757 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-20T09:21:27Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (203 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-20T09:21:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:21:27Z Disabled control of system clock
I20260620 09:21:27.930219 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:34577
--webserver_interface=127.4.59.62
--webserver_port=0
--builtin_ntp_servers=127.4.59.20:37477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:34577
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/testchainca.cert with env {}
W20260620 09:21:28.006973 9761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:28.007189 9761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:28.007225 9761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:28.008360 9761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:28.008424 9761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:28.008447 9761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:28.008473 9761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:28.008502 9761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 09:21:28.008526 9761 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 09:21:28.008555 9761 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260620 09:21:28.009994 9761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:37477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:34577
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:34577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:28.010252 9761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:28.010452 9761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 09:21:28.015062 9761 server_base.cc:1034] running on GCE node
W20260620 09:21:28.015024 9769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:28.015033 9767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:28.015044 9766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:28.015494 9761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:28.015718 9761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:28.016896 9761 hybrid_clock.cc:648] HybridClock initialized: now 1781947288016875 us; error 31 us; skew 500 ppm
I20260620 09:21:28.018224 9761 webserver.cc:460] Webserver started at http://127.4.59.62:41649/ using document root <none> and password file <none>
I20260620 09:21:28.018456 9761 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:28.018527 9761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:28.018657 9761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:28.019627 9761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "5b829f42c71b4a92bb656edba5762081"
format_stamp: "Formatted at 2026-06-20 09:21:28 on dist-test-slave-414w"
server_key: "667be2e08168c7b9880d82f0b6740cfb"
server_key_iv: "bf5c826af834ab527122b998080315cc"
server_key_version: "encryptionkey@0"
I20260620 09:21:28.019989 9761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "5b829f42c71b4a92bb656edba5762081"
format_stamp: "Formatted at 2026-06-20 09:21:28 on dist-test-slave-414w"
server_key: "667be2e08168c7b9880d82f0b6740cfb"
server_key_iv: "bf5c826af834ab527122b998080315cc"
server_key_version: "encryptionkey@0"
I20260620 09:21:28.021323 9761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:21:28.022012 9775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:28.022192 9761 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:28.022289 9761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "5b829f42c71b4a92bb656edba5762081"
format_stamp: "Formatted at 2026-06-20 09:21:28 on dist-test-slave-414w"
server_key: "667be2e08168c7b9880d82f0b6740cfb"
server_key_iv: "bf5c826af834ab527122b998080315cc"
server_key_version: "encryptionkey@0"
I20260620 09:21:28.022377 9761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:28.045389 9761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:28.045668 9761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:28.045823 9761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:28.049643 9761 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:34577
I20260620 09:21:28.049690 9827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:34577 every 8 connection(s)
I20260620 09:21:28.050024 9761 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:21:28.051120 9828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:28.054097 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 9761
I20260620 09:21:28.054214 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:21:28.054414 4332 external_mini_cluster.cc:1428] Setting key 4c51c8caab42ed93a227a8da9c5e26d1
I20260620 09:21:28.054486 9828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081: Bootstrap starting.
I20260620 09:21:28.055202 9828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:28.055550 9828 log.cc:824] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:28.056286 9828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081: No bootstrap required, opened a new log
I20260620 09:21:28.059139 9828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b829f42c71b4a92bb656edba5762081" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 34577 } }
I20260620 09:21:28.059315 9828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:28.059417 9828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b829f42c71b4a92bb656edba5762081, State: Initialized, Role: FOLLOWER
I20260620 09:21:28.059671 9831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:28.054858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59174 (local address 127.4.59.62:34577)
0620 09:21:28.055056 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:28.055063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:28.055087 (+ 24us) server_negotiation.cc:407] Connection header received
0620 09:21:28.055547 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:28.055553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:28.055630 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:28.055798 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:28.056007 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:28.056922 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:28.057794 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:28.058185 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:28.058317 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:28.058986 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:28.058989 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:28.058989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:21:28.059155 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:28.059186 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:21:28.059197 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:28.059278 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:28.059375 (+ 97us) server_negotiation.cc:299] Negotiation successful
0620 09:21:28.059453 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":91,"threads_started":1}
I20260620 09:21:28.059623 9828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b829f42c71b4a92bb656edba5762081" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 34577 } }
I20260620 09:21:28.059755 9828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:21:28.059793 9828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:21:28.059851 9828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:28.060586 9828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b829f42c71b4a92bb656edba5762081" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 34577 } }
I20260620 09:21:28.060623 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:21:28.060700 9828 leader_election.cc:302] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b829f42c71b4a92bb656edba5762081; no voters:
I20260620 09:21:28.060858 9828 leader_election.cc:288] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:21:28.060880 9833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:28.060986 9833 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [term 1 LEADER]: Becoming Leader. State: Replica: 5b829f42c71b4a92bb656edba5762081, State: Running, Role: LEADER
I20260620 09:21:28.061062 9833 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b829f42c71b4a92bb656edba5762081" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 34577 } }
I20260620 09:21:28.061100 9828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:28.061383 9833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b829f42c71b4a92bb656edba5762081. Latest consensus state: current_term: 1 leader_uuid: "5b829f42c71b4a92bb656edba5762081" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b829f42c71b4a92bb656edba5762081" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 34577 } } }
I20260620 09:21:28.061482 9833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:28.061396 9834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b829f42c71b4a92bb656edba5762081" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b829f42c71b4a92bb656edba5762081" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 34577 } } }
I20260620 09:21:28.061635 9834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:28.062297 9838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:21:28.062796 9838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:21:28.069540 9838 catalog_manager.cc:1347] Generated new cluster ID: efaa939f8fd34ca2b8ff46d144db4d0b
I20260620 09:21:28.069593 9838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:21:28.078677 9838 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:21:28.079403 9838 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:21:28.083747 9838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b829f42c71b4a92bb656edba5762081: Generated new TSK 0
I20260620 09:21:28.083920 9838 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:21:28.161595 9831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:28.092171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59186 (local address 127.4.59.62:34577)
0620 09:21:28.092216 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:28.092219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:28.092334 (+ 115us) server_negotiation.cc:407] Connection header received
0620 09:21:28.092396 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:28.092397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:28.092409 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:28.092485 (+ 76us) server_negotiation.cc:226] Negotiated authn=JWT
0620 09:21:28.092636 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:28.093465 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:28.094688 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:28.094998 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:28.095069 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:28.095256 (+ 187us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 09:21:28.161310 (+ 66054us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 09:21:28.161494 (+ 184us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59186: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":23}
W20260620 09:21:28.161669 9831 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59186: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260620 09:21:28.161818 9857 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:21:28.092121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:34577 (local address 127.0.0.1:59186)
0620 09:21:28.092264 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:28.092274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:28.092344 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:28.092498 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:28.092498 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:28.092500 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 09:21:28.092565 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:28.092568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:28.093547 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:28.093549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:28.094594 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:28.094598 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:28.095077 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:28.095085 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:28.095150 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:28.095156 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 09:21:28.161506 (+ 66350us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 09:21:28.161649 (+ 143us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.59.62:34577: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":98,"threads_started":1}
I20260620 09:21:28.162209 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 9761
2026-06-20T09:21:28Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (256 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (85 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (88 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:21:28 dist-test-slave-414w krb5kdc[9869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:21:28 dist-test-slave-414w krb5kdc[9869](info): set up 2 sockets
Jun 20 09:21:28 dist-test-slave-414w krb5kdc[9869](info): commencing operation
krb5kdc: starting...
W20260620 09:21:29.377722 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:21:29 dist-test-slave-414w krb5kdc[9869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:21:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:21:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:21:29.496245 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:45077
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:45077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:29.575276 9882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:29.575459 9882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:29.575480 9882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:29.576642 9882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:29.576683 9882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:29.576699 9882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:29.576710 9882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:29.576750 9882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:21:29.578212 9882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:45077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:45077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:29.578445 9882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:29.578680 9882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:29.583343 9888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:29.583350 9890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:29.583395 9887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:29.583585 9882 server_base.cc:1034] running on GCE node
I20260620 09:21:29.583810 9882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:29.584081 9882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:29.585204 9882 hybrid_clock.cc:648] HybridClock initialized: now 1781947289585183 us; error 38 us; skew 500 ppm
Jun 20 09:21:29 dist-test-slave-414w krb5kdc[9869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947289, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:29.593926 9882 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:21:29.594390 9882 webserver.cc:460] Webserver started at http://127.4.59.62:43169/ using document root <none> and password file <none>
I20260620 09:21:29.594596 9882 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:29.594640 9882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:29.594728 9882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:29.595731 9882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "a0cb8b2d769741d8bb2c5629c72ac6b8"
format_stamp: "Formatted at 2026-06-20 09:21:29 on dist-test-slave-414w"
server_key: "bfa8fd83243651877237dd8f2a97ade2"
server_key_iv: "057606b1713a9384c21ec6b8af1441bc"
server_key_version: "encryptionkey@0"
I20260620 09:21:29.596030 9882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "a0cb8b2d769741d8bb2c5629c72ac6b8"
format_stamp: "Formatted at 2026-06-20 09:21:29 on dist-test-slave-414w"
server_key: "bfa8fd83243651877237dd8f2a97ade2"
server_key_iv: "057606b1713a9384c21ec6b8af1441bc"
server_key_version: "encryptionkey@0"
I20260620 09:21:29.597306 9882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:21:29.597985 9897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:29.598181 9882 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:29.598292 9882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "a0cb8b2d769741d8bb2c5629c72ac6b8"
format_stamp: "Formatted at 2026-06-20 09:21:29 on dist-test-slave-414w"
server_key: "bfa8fd83243651877237dd8f2a97ade2"
server_key_iv: "057606b1713a9384c21ec6b8af1441bc"
server_key_version: "encryptionkey@0"
I20260620 09:21:29.598367 9882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:29.607304 9882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:29.609522 9882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:29.609678 9882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:29.613493 9882 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:45077
I20260620 09:21:29.613531 9949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:45077 every 8 connection(s)
I20260620 09:21:29.613890 9882 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:21:29.614725 9950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:29.616811 9950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8: Bootstrap starting.
I20260620 09:21:29.617384 9950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:29.617641 9950 log.cc:824] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:29.618234 9950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8: No bootstrap required, opened a new log
I20260620 09:21:29.619710 9950 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0cb8b2d769741d8bb2c5629c72ac6b8" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45077 } }
I20260620 09:21:29.619836 9950 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:29.619858 9950 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0cb8b2d769741d8bb2c5629c72ac6b8, State: Initialized, Role: FOLLOWER
I20260620 09:21:29.619956 9950 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0cb8b2d769741d8bb2c5629c72ac6b8" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45077 } }
I20260620 09:21:29.620013 9950 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:21:29.620033 9950 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:21:29.620064 9950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:29.620522 9950 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0cb8b2d769741d8bb2c5629c72ac6b8" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45077 } }
I20260620 09:21:29.620599 9950 leader_election.cc:302] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0cb8b2d769741d8bb2c5629c72ac6b8; no voters:
I20260620 09:21:29.620788 9950 leader_election.cc:288] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:21:29.620802 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 9882
I20260620 09:21:29.620847 9953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:29.620898 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:21:29.621057 9950 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:29.621080 4332 external_mini_cluster.cc:1428] Setting key 9582d7a90e1c7bad581df7a500bd87c8
I20260620 09:21:29.621654 9953 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [term 1 LEADER]: Becoming Leader. State: Replica: a0cb8b2d769741d8bb2c5629c72ac6b8, State: Running, Role: LEADER
I20260620 09:21:29.621809 9953 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0cb8b2d769741d8bb2c5629c72ac6b8" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45077 } }
I20260620 09:21:29.622262 9956 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0cb8b2d769741d8bb2c5629c72ac6b8. Latest consensus state: current_term: 1 leader_uuid: "a0cb8b2d769741d8bb2c5629c72ac6b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0cb8b2d769741d8bb2c5629c72ac6b8" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45077 } } }
I20260620 09:21:29.622349 9956 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:29.622408 9954 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0cb8b2d769741d8bb2c5629c72ac6b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0cb8b2d769741d8bb2c5629c72ac6b8" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45077 } } }
I20260620 09:21:29.622475 9954 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8 [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:29.622874 9967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:21:29.623387 9967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:21:29.625007 9967 catalog_manager.cc:1347] Generated new cluster ID: 75842cc4514545eda6f9e9fe42461dcd
I20260620 09:21:29.625063 9967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 09:21:29 dist-test-slave-414w krb5kdc[9869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:29.633489 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:29.621704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59512 (local address 127.4.59.62:45077)
0620 09:21:29.621861 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:29.621869 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:29.621902 (+ 33us) server_negotiation.cc:407] Connection header received
0620 09:21:29.622010 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:29.622035 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:29.622092 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:29.623356 (+ 1264us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:29.623463 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:29.625077 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:29.626235 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:29.626950 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:29.627064 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:29.629872 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:29.629878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:29.629879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:29.629895 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:29.631828 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:29.632210 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:29.632213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:29.632214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:29.632319 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:29.632518 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:29.632519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:29.632519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:29.632838 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:29.633011 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:29.633080 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 09:21:29.633258 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260620 09:21:29.640965 9967 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:21:29.641516 9967 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:21:29.644412 9967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0cb8b2d769741d8bb2c5629c72ac6b8: Generated new TSK 0
I20260620 09:21:29.644590 9967 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:21:29.670241 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45077
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:29.749955 9976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:29.750161 9976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:29.750197 9976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:29.751320 9976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:29.751382 9976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:29.751430 9976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:21:29.752882 9976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:29.753134 9976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:29.753363 9976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 09:21:29.758258 9976 server_base.cc:1034] running on GCE node
W20260620 09:21:29.758242 9982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:29.758239 9981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:29.758222 9984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:29.758586 9976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:29.758832 9976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:29.759970 9976 hybrid_clock.cc:648] HybridClock initialized: now 1781947289759949 us; error 37 us; skew 500 ppm
Jun 20 09:21:29 dist-test-slave-414w krb5kdc[9869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947289, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:29.762413 9976 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:21:29.762851 9976 webserver.cc:460] Webserver started at http://127.4.59.1:36193/ using document root <none> and password file <none>
I20260620 09:21:29.763062 9976 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:29.763119 9976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:29.763206 9976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:29.764135 9976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "d1c3ffc3fa964865af177b6024126782"
format_stamp: "Formatted at 2026-06-20 09:21:29 on dist-test-slave-414w"
server_key: "1ace4b7e947e2d048f9de37748555d1e"
server_key_iv: "de24418ef216f3e7d149d5087aa0d173"
server_key_version: "encryptionkey@0"
I20260620 09:21:29.764423 9976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "d1c3ffc3fa964865af177b6024126782"
format_stamp: "Formatted at 2026-06-20 09:21:29 on dist-test-slave-414w"
server_key: "1ace4b7e947e2d048f9de37748555d1e"
server_key_iv: "de24418ef216f3e7d149d5087aa0d173"
server_key_version: "encryptionkey@0"
I20260620 09:21:29.765743 9976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:21:29.766462 9991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:29.766719 9976 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:29.766796 9976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "d1c3ffc3fa964865af177b6024126782"
format_stamp: "Formatted at 2026-06-20 09:21:29 on dist-test-slave-414w"
server_key: "1ace4b7e947e2d048f9de37748555d1e"
server_key_iv: "de24418ef216f3e7d149d5087aa0d173"
server_key_version: "encryptionkey@0"
I20260620 09:21:29.766850 9976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:29.783620 9976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:29.786051 9976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:29.786170 9976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:29.786355 9976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:29.786675 9976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:29.786713 9976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:29.786758 9976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:29.786787 9976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:29.793280 9976 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:33253
I20260620 09:21:29.793329 10104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:33253 every 8 connection(s)
I20260620 09:21:29.793749 9976 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:21:29.794641 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 9976
I20260620 09:21:29.794754 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:21:29.794960 4332 external_mini_cluster.cc:1428] Setting key 30e46154be54072ea5b7c95d627f7734
Jun 20 09:21:29 dist-test-slave-414w krb5kdc[9869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947289, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
I20260620 09:21:29.807879 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:29.796887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:42037 (local address 127.4.59.62:45077)
0620 09:21:29.797393 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:29.797396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:29.797407 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:21:29.798465 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:29.798467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:29.798484 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:29.798524 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:29.799298 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:29.800192 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:29.802131 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:29.802489 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:29.802578 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:29.804845 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:29.804847 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:29.804848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:29.804856 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:29.806469 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:29.806862 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:29.806863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:29.806864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:29.806926 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:29.807192 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:29.807193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:29.807193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:29.807313 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:29.807398 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:29.807692 (+ 294us) server_negotiation.cc:299] Negotiation successful
0620 09:21:29.807777 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474}
I20260620 09:21:29.808117 10107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:29.796616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45077 (local address 127.4.59.1:42037)
0620 09:21:29.796822 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:29.796906 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:29.798324 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:29.798734 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:29.798738 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:29.799004 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:29.799199 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:29.799203 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:29.800332 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:29.800333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:29.802043 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:29.802046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:29.802592 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:29.802593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:29.802736 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:29.803309 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:29.803318 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:29.804743 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:29.806556 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:29.806562 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:29.806564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:29.806790 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:29.806987 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:29.806988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:29.806988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:29.807103 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:29.807388 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:29.807395 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:29.807532 (+ 137us) client_negotiation.cc:769] Sending connection context
0620 09:21:29.807663 (+ 131us) client_negotiation.cc:240] Negotiation successful
0620 09:21:29.807830 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":88,"threads_started":1}
I20260620 09:21:29.808817 10105 heartbeater.cc:347] Connected to a master server at 127.4.59.62:45077
I20260620 09:21:29.808930 10105 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:29.809199 10105 heartbeater.cc:511] Master 127.4.59.62:45077 requested a full tablet report, sending...
I20260620 09:21:29.809773 9914 ts_manager.cc:194] Registered new tserver with Master: d1c3ffc3fa964865af177b6024126782 (127.4.59.1:33253)
I20260620 09:21:29.810428 9914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:42037
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:21:29.821508 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45077
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:29.902209 10110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:29.902390 10110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:29.902411 10110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:29.903476 10110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:29.903509 10110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:29.903533 10110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:21:29.905596 10110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:29.905782 10110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:29.905949 10110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:29.910831 10116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:29.910951 10115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:29.911070 10118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:29.911085 10110 server_base.cc:1034] running on GCE node
I20260620 09:21:29.911350 10110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:29.911580 10110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:29.912751 10110 hybrid_clock.cc:648] HybridClock initialized: now 1781947289912736 us; error 31 us; skew 500 ppm
Jun 20 09:21:29 dist-test-slave-414w krb5kdc[9869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947289, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:29.915359 10110 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:21:29.915839 10110 webserver.cc:460] Webserver started at http://127.4.59.2:42467/ using document root <none> and password file <none>
I20260620 09:21:29.916085 10110 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:29.916152 10110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:29.916282 10110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:29.917347 10110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "593b4fd722e84cc588e52af189da943a"
format_stamp: "Formatted at 2026-06-20 09:21:29 on dist-test-slave-414w"
server_key: "a2bb5eb5e73610e916d01ccab95378f6"
server_key_iv: "ae9e9c8b987ca7651680d82ab79c9c25"
server_key_version: "encryptionkey@0"
I20260620 09:21:29.917817 10110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "593b4fd722e84cc588e52af189da943a"
format_stamp: "Formatted at 2026-06-20 09:21:29 on dist-test-slave-414w"
server_key: "a2bb5eb5e73610e916d01ccab95378f6"
server_key_iv: "ae9e9c8b987ca7651680d82ab79c9c25"
server_key_version: "encryptionkey@0"
I20260620 09:21:29.919246 10110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:21:29.920264 10125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:29.920522 10110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:29.920652 10110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "593b4fd722e84cc588e52af189da943a"
format_stamp: "Formatted at 2026-06-20 09:21:29 on dist-test-slave-414w"
server_key: "a2bb5eb5e73610e916d01ccab95378f6"
server_key_iv: "ae9e9c8b987ca7651680d82ab79c9c25"
server_key_version: "encryptionkey@0"
I20260620 09:21:29.920742 10110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:29.934370 10110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:29.936355 10110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:29.936517 10110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:29.936740 10110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:29.937040 10110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:29.937101 10110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:29.937197 10110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:29.937244 10110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:29.942886 10110 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:44425
I20260620 09:21:29.942935 10238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:44425 every 8 connection(s)
I20260620 09:21:29.943225 10110 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:21:29.945813 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 10110
I20260620 09:21:29.945926 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:21:29.946103 4332 external_mini_cluster.cc:1428] Setting key 8891749fcd1c3ac33cfa36e0937952dc
Jun 20 09:21:29 dist-test-slave-414w krb5kdc[9869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947289, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:29.953028 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:29.943870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:54183 (local address 127.4.59.62:45077)
0620 09:21:29.943956 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:29.943958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:29.944149 (+ 191us) server_negotiation.cc:407] Connection header received
0620 09:21:29.945031 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:29.945032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:29.945043 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:29.945095 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:29.945656 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:29.946288 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:29.947439 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:29.947775 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:29.947847 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:29.950088 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:29.950090 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:29.950090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:29.950098 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:29.951720 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:29.952090 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:29.952092 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:29.952092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:29.952155 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:29.952378 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:29.952379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:29.952379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:29.952518 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:29.952599 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:29.952832 (+ 233us) server_negotiation.cc:299] Negotiation successful
0620 09:21:29.952913 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 09:21:29.953050 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:29.943821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45077 (local address 127.4.59.2:54183)
0620 09:21:29.943995 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:29.944056 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:29.944944 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:29.945161 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:29.945165 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:29.945401 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:29.945587 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:29.945590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:29.946360 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:29.946361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:29.947349 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:29.947351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:29.947861 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:29.947862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:29.947997 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:29.948460 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:29.948471 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:29.949993 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:29.951801 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:29.951807 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:29.951809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:29.952015 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:29.952225 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:29.952226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:29.952226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:29.952314 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:29.952602 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:29.952605 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:29.952709 (+ 104us) client_negotiation.cc:769] Sending connection context
0620 09:21:29.952788 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 09:21:29.952901 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260620 09:21:29.953490 10239 heartbeater.cc:347] Connected to a master server at 127.4.59.62:45077
I20260620 09:21:29.953604 10239 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:29.953766 10239 heartbeater.cc:511] Master 127.4.59.62:45077 requested a full tablet report, sending...
I20260620 09:21:29.954136 9914 ts_manager.cc:194] Registered new tserver with Master: 593b4fd722e84cc588e52af189da943a (127.4.59.2:44425)
I20260620 09:21:29.954665 9914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:54183
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:21:29.970095 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45077
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:30.048728 10244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:30.048923 10244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:30.048945 10244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:30.050117 10244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:30.050156 10244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:30.050182 10244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:21:30.051622 10244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:30.051868 10244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:30.052094 10244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:30.056871 10252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:30.056883 10249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:30.056882 10250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:30.057114 10244 server_base.cc:1034] running on GCE node
I20260620 09:21:30.057356 10244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:30.057554 10244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:30.058692 10244 hybrid_clock.cc:648] HybridClock initialized: now 1781947290058686 us; error 31 us; skew 500 ppm
Jun 20 09:21:30 dist-test-slave-414w krb5kdc[9869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947290, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:30.061241 10244 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:21:30.061609 10244 webserver.cc:460] Webserver started at http://127.4.59.3:40495/ using document root <none> and password file <none>
I20260620 09:21:30.061800 10244 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:30.061839 10244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:30.061921 10244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:30.062860 10244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "cd9dda046e584e94a1f3ad5d009f6164"
format_stamp: "Formatted at 2026-06-20 09:21:30 on dist-test-slave-414w"
server_key: "4be4c330fcfa4e1ad695d21d68c268c7"
server_key_iv: "585f39f2a87f0c552129b60159ee11da"
server_key_version: "encryptionkey@0"
I20260620 09:21:30.063158 10244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "cd9dda046e584e94a1f3ad5d009f6164"
format_stamp: "Formatted at 2026-06-20 09:21:30 on dist-test-slave-414w"
server_key: "4be4c330fcfa4e1ad695d21d68c268c7"
server_key_iv: "585f39f2a87f0c552129b60159ee11da"
server_key_version: "encryptionkey@0"
I20260620 09:21:30.064391 10244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:21:30.065107 10259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:30.065338 10244 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:30.065440 10244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "cd9dda046e584e94a1f3ad5d009f6164"
format_stamp: "Formatted at 2026-06-20 09:21:30 on dist-test-slave-414w"
server_key: "4be4c330fcfa4e1ad695d21d68c268c7"
server_key_iv: "585f39f2a87f0c552129b60159ee11da"
server_key_version: "encryptionkey@0"
I20260620 09:21:30.065522 10244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:30.082180 10244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:30.084496 10244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:30.084657 10244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:30.084893 10244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:30.085271 10244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:30.085325 10244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:30.085367 10244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:30.085393 10244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:30.091189 10244 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:37901
I20260620 09:21:30.091231 10372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:37901 every 8 connection(s)
I20260620 09:21:30.091539 10244 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:21:30.094559 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 10244
I20260620 09:21:30.094691 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:21:30.094877 4332 external_mini_cluster.cc:1428] Setting key 61cee91ad6d06430fcbff83742e842ed
Jun 20 09:21:30 dist-test-slave-414w krb5kdc[9869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947290, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:21:30.100957 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:30.092230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:38669 (local address 127.4.59.62:45077)
0620 09:21:30.092276 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:30.092278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:30.092526 (+ 248us) server_negotiation.cc:407] Connection header received
0620 09:21:30.093542 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:30.093544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:30.093555 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:30.093608 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:30.094207 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:30.094733 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:30.096072 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:30.096262 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:30.096329 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:30.098356 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:30.098357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:30.098358 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:30.098364 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:30.099642 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:30.100005 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:30.100006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:30.100006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:30.100060 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:30.100325 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:30.100325 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:30.100326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:30.100423 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:30.100488 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:30.100760 (+ 272us) server_negotiation.cc:299] Negotiation successful
0620 09:21:30.100870 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:21:30.101019 10375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:30.092232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45077 (local address 127.4.59.3:38669)
0620 09:21:30.092404 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:30.092439 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:30.093437 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:30.093689 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:30.093693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:30.093933 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:30.094114 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:30.094118 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:30.094860 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:30.094861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:30.095950 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:30.095952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:30.096340 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:30.096341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:30.096433 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:30.096932 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:30.096940 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:30.098277 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:30.099720 (+ 1443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:30.099725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:30.099727 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:30.099931 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:30.100152 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:30.100153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:30.100153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:30.100260 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:30.100487 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:30.100491 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:30.100603 (+ 112us) client_negotiation.cc:769] Sending connection context
0620 09:21:30.100707 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 09:21:30.100825 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
I20260620 09:21:30.101511 10373 heartbeater.cc:347] Connected to a master server at 127.4.59.62:45077
I20260620 09:21:30.101601 10373 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:30.101773 10373 heartbeater.cc:511] Master 127.4.59.62:45077 requested a full tablet report, sending...
I20260620 09:21:30.102102 9914 ts_manager.cc:194] Registered new tserver with Master: cd9dda046e584e94a1f3ad5d009f6164 (127.4.59.3:37901)
I20260620 09:21:30.102468 9914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:38669
I20260620 09:21:30.107113 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/test-admin.keytab.
Jun 20 09:21:30 dist-test-slave-414w krb5kdc[9869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:30.122466 4332 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260620 09:21:30.124224 10380 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:21:30.123427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54142 (local address 127.4.59.1:33253)
0620 09:21:30.123646 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:30.123657 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:30.123684 (+ 27us) server_negotiation.cc:407] Connection header received
0620 09:21:30.123745 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:30.123747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:30.123760 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:30.123828 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:30.124134 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54142: BlockingRecv error: recv got EOF from 127.0.0.1:54142 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":112,"threads_started":1}
W20260620 09:21:30.124286 4332 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/krb5cc' of type FILE
Jun 20 09:21:30 dist-test-slave-414w krb5kdc[9869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:30.125349 4332 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 09:21:30 dist-test-slave-414w krb5kdc[9869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:30.126328 4332 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 09:21:30 dist-test-slave-414w krb5kdc[9869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.1@KRBTEST.COM
I20260620 09:21:30.134276 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:30.126899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54144 (local address 127.4.59.1:33253)
0620 09:21:30.126936 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:30.126937 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:30.126995 (+ 58us) server_negotiation.cc:407] Connection header received
0620 09:21:30.127045 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:30.127046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:30.127055 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:30.127107 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:30.127470 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:30.128189 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:30.128962 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:30.129403 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:30.129464 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:30.131511 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:30.131518 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:30.131518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:30.131528 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:30.133293 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:30.133646 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:30.133648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:30.133648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:30.133700 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:30.133865 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:30.133866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:30.133866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:30.133984 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:30.134097 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:30.134151 (+ 54us) server_negotiation.cc:299] Negotiation successful
0620 09:21:30.134206 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260620 09:21:30.135955 10380 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:21:30.135427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54158 (local address 127.4.59.1:33253)
0620 09:21:30.135462 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:30.135464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:30.135527 (+ 63us) server_negotiation.cc:407] Connection header received
0620 09:21:30.135555 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:30.135555 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:30.135563 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:30.135609 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:30.135905 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54158: BlockingRecv error: recv got EOF from 127.0.0.1:54158 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260620 09:21:30.136050 4332 init.cc:244] Error while opening credential cache '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/krb5cc)
Jun 20 09:21:30 dist-test-slave-414w krb5kdc[9869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:30.136938 4332 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 09:21:30 dist-test-slave-414w krb5kdc[9869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:30.137815 4332 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 09:21:30 dist-test-slave-414w krb5kdc[9869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.1@KRBTEST.COM
I20260620 09:21:30.144084 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:30.138291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54160 (local address 127.4.59.1:33253)
0620 09:21:30.138350 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:30.138351 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:30.138373 (+ 22us) server_negotiation.cc:407] Connection header received
0620 09:21:30.138419 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:30.138420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:30.138427 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:30.138475 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:30.138794 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:30.139250 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:30.139895 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:30.140078 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:30.140132 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:30.141950 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:30.141951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:30.141951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:30.141957 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:30.143066 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:30.143377 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:30.143378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:30.143378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:30.143425 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:30.143579 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:30.143594 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:30.143594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:30.143674 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:30.143741 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:30.143898 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 09:21:30.143952 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260620 09:21:30.145709 10380 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:21:30.145100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54176 (local address 127.4.59.1:33253)
0620 09:21:30.145179 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:30.145181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:30.145276 (+ 95us) server_negotiation.cc:407] Connection header received
0620 09:21:30.145346 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:30.145347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:30.145355 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:30.145410 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:30.145661 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54176: BlockingRecv error: recv got EOF from 127.0.0.1:54176 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260620 09:21:30.145793 4332 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781947262065055-4332-0/krb5kdc/krb5cc' of type FILE
Jun 20 09:21:30 dist-test-slave-414w krb5kdc[9869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:30.146641 4332 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 09:21:30 dist-test-slave-414w krb5kdc[9869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:30.147430 4332 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 09:21:30 dist-test-slave-414w krb5kdc[9869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.1@KRBTEST.COM
I20260620 09:21:30.153805 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:30.147891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54180 (local address 127.4.59.1:33253)
0620 09:21:30.147949 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:30.147950 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:30.147959 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:21:30.148004 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:30.148004 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:30.148011 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:30.148058 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:30.148358 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:30.148791 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:30.149439 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:30.149621 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:30.149672 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:30.151537 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:30.151538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:30.151538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:30.151544 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:30.152707 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:30.153016 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:30.153017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:30.153017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:30.153069 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:30.153354 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:30.153354 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:30.153354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:30.153442 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:30.153520 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:30.153606 (+ 86us) server_negotiation.cc:299] Negotiation successful
0620 09:21:30.153668 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 09:21:30.154378 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 9976
I20260620 09:21:30.159961 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 10110
I20260620 09:21:30.164620 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 10244
I20260620 09:21:30.168421 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 9882
2026-06-20T09:21:30Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1833 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:21:30 dist-test-slave-414w krb5kdc[10393](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:21:30 dist-test-slave-414w krb5kdc[10393](info): set up 2 sockets
Jun 20 09:21:30 dist-test-slave-414w krb5kdc[10393](info): commencing operation
krb5kdc: starting...
W20260620 09:21:31.209766 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:21:31 dist-test-slave-414w krb5kdc[10393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:21:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:21:31Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.62.keytab.
Entry for principal oryx/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.62.keytab.
I20260620 09:21:31.321441 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:46555
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.62.keytab
--principal=oryx/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:46555
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:31.400012 10406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:31.400200 10406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:31.400223 10406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:31.401317 10406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:31.401351 10406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:31.401365 10406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:31.401379 10406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:31.401391 10406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:21:31.402750 10406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:46555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.62.keytab
--principal=oryx/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:46555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:31.402926 10406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:31.403080 10406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:31.407822 10411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:31.407879 10412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:31.407871 10414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:31.408008 10406 server_base.cc:1034] running on GCE node
I20260620 09:21:31.408283 10406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:31.408574 10406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:31.409729 10406 hybrid_clock.cc:648] HybridClock initialized: now 1781947291409713 us; error 36 us; skew 500 ppm
Jun 20 09:21:31 dist-test-slave-414w krb5kdc[10393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947291, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:31.412034 10406 init.cc:377] Logged in from keytab as oryx/127.4.59.62@KRBTEST.COM (short username oryx)
I20260620 09:21:31.412441 10406 webserver.cc:460] Webserver started at http://127.4.59.62:33413/ using document root <none> and password file <none>
I20260620 09:21:31.412665 10406 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:31.412732 10406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:31.412882 10406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:31.413950 10406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "0990b69bade74124ab980ca951f7a9ad"
format_stamp: "Formatted at 2026-06-20 09:21:31 on dist-test-slave-414w"
server_key: "c24fe16a45c0846fe6a72ccc033b3854"
server_key_iv: "1d483f5d15a6f8734d1ead93c7dd8353"
server_key_version: "encryptionkey@0"
I20260620 09:21:31.414296 10406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "0990b69bade74124ab980ca951f7a9ad"
format_stamp: "Formatted at 2026-06-20 09:21:31 on dist-test-slave-414w"
server_key: "c24fe16a45c0846fe6a72ccc033b3854"
server_key_iv: "1d483f5d15a6f8734d1ead93c7dd8353"
server_key_version: "encryptionkey@0"
I20260620 09:21:31.415604 10406 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 09:21:31.416500 10421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:31.416787 10406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:21:31.416891 10406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "0990b69bade74124ab980ca951f7a9ad"
format_stamp: "Formatted at 2026-06-20 09:21:31 on dist-test-slave-414w"
server_key: "c24fe16a45c0846fe6a72ccc033b3854"
server_key_iv: "1d483f5d15a6f8734d1ead93c7dd8353"
server_key_version: "encryptionkey@0"
I20260620 09:21:31.416975 10406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:31.425459 10406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:31.427354 10406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:31.427520 10406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:31.431324 10406 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:46555
I20260620 09:21:31.431375 10473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:46555 every 8 connection(s)
I20260620 09:21:31.431722 10406 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:21:31.432744 10474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:31.434903 10474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad: Bootstrap starting.
I20260620 09:21:31.435398 10474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:31.435617 10474 log.cc:824] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:31.435976 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 10406
I20260620 09:21:31.436110 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:21:31.436311 4332 external_mini_cluster.cc:1428] Setting key e865cb406feaae45cc8d06e62911127e
I20260620 09:21:31.436348 10474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad: No bootstrap required, opened a new log
I20260620 09:21:31.438252 10474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } }
I20260620 09:21:31.438401 10474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:31.438445 10474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0990b69bade74124ab980ca951f7a9ad, State: Initialized, Role: FOLLOWER
I20260620 09:21:31.438573 10474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } }
I20260620 09:21:31.438655 10474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:21:31.438690 10474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:21:31.438751 10474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:31.439672 10474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } }
I20260620 09:21:31.439764 10474 leader_election.cc:302] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0990b69bade74124ab980ca951f7a9ad; no voters:
I20260620 09:21:31.439946 10474 leader_election.cc:288] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:21:31.439968 10479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:31.440069 10479 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [term 1 LEADER]: Becoming Leader. State: Replica: 0990b69bade74124ab980ca951f7a9ad, State: Running, Role: LEADER
I20260620 09:21:31.440147 10479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } }
I20260620 09:21:31.440194 10474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:31.440454 10479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0990b69bade74124ab980ca951f7a9ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } }
I20260620 09:21:31.440517 10479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:31.440649 10479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0990b69bade74124ab980ca951f7a9ad. Latest consensus state: current_term: 1 leader_uuid: "0990b69bade74124ab980ca951f7a9ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } }
I20260620 09:21:31.440707 10479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:31.441334 10483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:21:31.441875 10483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:21:31.443399 10483 catalog_manager.cc:1347] Generated new cluster ID: 3d88ecb57cd148b887151187edc1a264
I20260620 09:21:31.443452 10483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 09:21:31 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM
I20260620 09:21:31.448024 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:31.436817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60972 (local address 127.4.59.62:46555)
0620 09:21:31.436987 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:31.436995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:31.437029 (+ 34us) server_negotiation.cc:407] Connection header received
0620 09:21:31.438240 (+ 1211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:31.438247 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:31.438301 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:31.438446 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:31.439169 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:31.440244 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:31.441351 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:31.441961 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:31.442059 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:31.444628 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:31.444631 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:31.444632 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:31.444642 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:31.446325 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:31.446756 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:31.446759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:31.446759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:31.446869 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:31.447086 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:31.447086 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:31.447087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:31.447320 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:31.447483 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:31.447576 (+ 93us) server_negotiation.cc:299] Negotiation successful
0620 09:21:31.447712 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260620 09:21:31.450623 10483 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:21:31.451102 10483 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:21:31.454582 10483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad: Generated new TSK 0
I20260620 09:21:31.454744 10483 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.1.keytab.
Entry for principal oryx/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.1.keytab.
I20260620 09:21:31.481925 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:46555
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.1.keytab
--principal=oryx/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:31.570287 10500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:31.570469 10500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:31.570492 10500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:31.571627 10500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:31.571676 10500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:31.571702 10500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:21:31.573107 10500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.1.keytab
--principal=oryx/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:46555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:31.573365 10500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:31.573585 10500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:31.578684 10506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:31.578743 10505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:31.578974 10508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:31.579007 10500 server_base.cc:1034] running on GCE node
I20260620 09:21:31.579211 10500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:31.579468 10500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:31.580629 10500 hybrid_clock.cc:648] HybridClock initialized: now 1781947291580627 us; error 36 us; skew 500 ppm
Jun 20 09:21:31 dist-test-slave-414w krb5kdc[10393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947291, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:31.583357 10500 init.cc:377] Logged in from keytab as oryx/127.4.59.1@KRBTEST.COM (short username oryx)
I20260620 09:21:31.583827 10500 webserver.cc:460] Webserver started at http://127.4.59.1:34041/ using document root <none> and password file <none>
I20260620 09:21:31.584052 10500 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:31.584120 10500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:31.584254 10500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:31.585333 10500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "6ab00061f9c843e0bab91896d7649893"
format_stamp: "Formatted at 2026-06-20 09:21:31 on dist-test-slave-414w"
server_key: "27c9683403335b481028eac1f340ec5c"
server_key_iv: "6d0cf9f2b79a2ad6f42044703dbb7216"
server_key_version: "encryptionkey@0"
I20260620 09:21:31.585687 10500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "6ab00061f9c843e0bab91896d7649893"
format_stamp: "Formatted at 2026-06-20 09:21:31 on dist-test-slave-414w"
server_key: "27c9683403335b481028eac1f340ec5c"
server_key_iv: "6d0cf9f2b79a2ad6f42044703dbb7216"
server_key_version: "encryptionkey@0"
I20260620 09:21:31.587013 10500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 09:21:31.587767 10515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:31.587927 10500 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:31.588028 10500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "6ab00061f9c843e0bab91896d7649893"
format_stamp: "Formatted at 2026-06-20 09:21:31 on dist-test-slave-414w"
server_key: "27c9683403335b481028eac1f340ec5c"
server_key_iv: "6d0cf9f2b79a2ad6f42044703dbb7216"
server_key_version: "encryptionkey@0"
I20260620 09:21:31.588114 10500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:31.614919 10500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:31.616602 10500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:31.616761 10500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:31.617002 10500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:31.617336 10500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:31.617390 10500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:31.617430 10500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:31.617457 10500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:31.623008 10500 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:34235
I20260620 09:21:31.623072 10628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:34235 every 8 connection(s)
I20260620 09:21:31.623378 10500 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:21:31.627202 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 10500
I20260620 09:21:31.627308 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:21:31.627487 4332 external_mini_cluster.cc:1428] Setting key 0de3421e291971623a02c0ebd96ac676
Jun 20 09:21:31 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947291, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM
I20260620 09:21:31.633023 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:31.624067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:34379 (local address 127.4.59.62:46555)
0620 09:21:31.624126 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:31.624128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:31.624349 (+ 221us) server_negotiation.cc:407] Connection header received
0620 09:21:31.624846 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:31.624847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:31.624862 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:31.624925 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:31.625904 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:31.626399 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:31.627405 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:31.627676 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:31.627740 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:31.630020 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:31.630022 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:31.630022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:31.630029 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:31.631653 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:31.632082 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:31.632083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:31.632084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:31.632145 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:31.632390 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:31.632391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:31.632391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:31.632499 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:31.632569 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:31.632848 (+ 279us) server_negotiation.cc:299] Negotiation successful
0620 09:21:31.632932 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 09:21:31.633205 10631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:31.624061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:34379)
0620 09:21:31.624234 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:31.624266 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:31.624738 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:31.624990 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:31.624994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:31.625629 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:31.625807 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:31.625811 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:31.626491 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:31.626492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:31.627270 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:31.627272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:31.627762 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:31.627763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:31.627912 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:31.628435 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:31.628449 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:31.629911 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:31.631764 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:31.631771 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:31.631773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:31.632001 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:31.632212 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:31.632213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:31.632213 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:31.632319 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:31.632574 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:31.632580 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:31.632703 (+ 123us) client_negotiation.cc:769] Sending connection context
0620 09:21:31.632810 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 09:21:31.632961 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
I20260620 09:21:31.633795 10629 heartbeater.cc:347] Connected to a master server at 127.4.59.62:46555
I20260620 09:21:31.633872 10629 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:31.634059 10629 heartbeater.cc:511] Master 127.4.59.62:46555 requested a full tablet report, sending...
I20260620 09:21:31.634608 10438 ts_manager.cc:194] Registered new tserver with Master: 6ab00061f9c843e0bab91896d7649893 (127.4.59.1:34235)
WARNING: no policy specified for oryx/127.4.59.2@KRBTEST.COM; defaulting to no policy
I20260620 09:21:31.635233 10438 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:34379
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.2.keytab.
Entry for principal oryx/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.2.keytab.
I20260620 09:21:31.652221 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:46555
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.2.keytab
--principal=oryx/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:31.730863 10634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:31.731055 10634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:31.731077 10634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:31.732200 10634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:31.732240 10634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:31.732266 10634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:21:31.733733 10634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.2.keytab
--principal=oryx/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:46555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:31.733961 10634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:31.734180 10634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:31.739132 10642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:31.739135 10640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:31.739130 10639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:31.739197 10634 server_base.cc:1034] running on GCE node
I20260620 09:21:31.739499 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:31.739779 10634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:31.740934 10634 hybrid_clock.cc:648] HybridClock initialized: now 1781947291740923 us; error 46 us; skew 500 ppm
Jun 20 09:21:31 dist-test-slave-414w krb5kdc[10393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947291, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:31.743412 10634 init.cc:377] Logged in from keytab as oryx/127.4.59.2@KRBTEST.COM (short username oryx)
I20260620 09:21:31.743834 10634 webserver.cc:460] Webserver started at http://127.4.59.2:46597/ using document root <none> and password file <none>
I20260620 09:21:31.744045 10634 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:31.744086 10634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:31.744221 10634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:31.745285 10634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "e70728dae33f4444984b8f40f36d2d27"
format_stamp: "Formatted at 2026-06-20 09:21:31 on dist-test-slave-414w"
server_key: "a6612d99f94f9659d84717db811f52da"
server_key_iv: "5ad2df66e6bd7c3b0d7dae238a23d925"
server_key_version: "encryptionkey@0"
I20260620 09:21:31.745623 10634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "e70728dae33f4444984b8f40f36d2d27"
format_stamp: "Formatted at 2026-06-20 09:21:31 on dist-test-slave-414w"
server_key: "a6612d99f94f9659d84717db811f52da"
server_key_iv: "5ad2df66e6bd7c3b0d7dae238a23d925"
server_key_version: "encryptionkey@0"
I20260620 09:21:31.746903 10634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:31.747612 10649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:31.747808 10634 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:31.747895 10634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "e70728dae33f4444984b8f40f36d2d27"
format_stamp: "Formatted at 2026-06-20 09:21:31 on dist-test-slave-414w"
server_key: "a6612d99f94f9659d84717db811f52da"
server_key_iv: "5ad2df66e6bd7c3b0d7dae238a23d925"
server_key_version: "encryptionkey@0"
I20260620 09:21:31.747946 10634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:31.756477 10634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:31.758354 10634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:31.758463 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:31.758621 10634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:31.759061 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:31.759101 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:31.759124 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:31.759137 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:31.764739 10634 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:45543
I20260620 09:21:31.764812 10762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:45543 every 8 connection(s)
I20260620 09:21:31.765153 10634 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:21:31.766331 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 10634
I20260620 09:21:31.766440 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:21:31.766618 4332 external_mini_cluster.cc:1428] Setting key 8c4b07b3d365bc73f26d3df1ab3578f0
Jun 20 09:21:31 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947291, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.4.59.3@KRBTEST.COM; defaulting to no policy
I20260620 09:21:31.775599 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:31.765888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:57473 (local address 127.4.59.62:46555)
0620 09:21:31.765953 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:31.765955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:31.766153 (+ 198us) server_negotiation.cc:407] Connection header received
0620 09:21:31.766721 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:31.766722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:31.766736 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:31.766800 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:31.767849 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:31.768379 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:31.769510 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:31.769839 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:31.769916 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:31.772381 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:31.772382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:31.772382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:31.772391 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:31.774071 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:31.774493 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:31.774494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:31.774495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:31.774559 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:31.774842 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:31.774843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:31.774843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:31.774995 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:31.775071 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:31.775453 (+ 382us) server_negotiation.cc:299] Negotiation successful
0620 09:21:31.775531 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 09:21:31.775826 10765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:31.765850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:57473)
0620 09:21:31.766022 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:31.766064 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:31.766613 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:31.766890 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:31.766894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:31.767559 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:31.767743 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:31.767747 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:31.768484 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:31.768486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:31.769417 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:31.769420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:31.769943 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:31.769945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:31.770052 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:31.770631 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:31.770639 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:31.772281 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:31.774170 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:31.774176 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:31.774178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:31.774408 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:31.774656 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:31.774657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:31.774657 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:31.774762 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:31.775108 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:31.775114 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:31.775284 (+ 170us) client_negotiation.cc:769] Sending connection context
0620 09:21:31.775423 (+ 139us) client_negotiation.cc:240] Negotiation successful
0620 09:21:31.775597 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260620 09:21:31.776454 10763 heartbeater.cc:347] Connected to a master server at 127.4.59.62:46555
I20260620 09:21:31.776569 10763 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:31.776729 10763 heartbeater.cc:511] Master 127.4.59.62:46555 requested a full tablet report, sending...
I20260620 09:21:31.777213 10438 ts_manager.cc:194] Registered new tserver with Master: e70728dae33f4444984b8f40f36d2d27 (127.4.59.2:45543)
I20260620 09:21:31.777654 10438 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:57473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.3.keytab.
Entry for principal oryx/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.3.keytab.
I20260620 09:21:31.792198 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:46555
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.3.keytab
--principal=oryx/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:31.877632 10768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:31.877840 10768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:31.877863 10768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:31.879019 10768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:31.879060 10768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:31.879084 10768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:21:31.880558 10768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.3.keytab
--principal=oryx/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:46555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:31.880829 10768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:31.881062 10768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:31.886109 10774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:31.886379 10768 server_base.cc:1034] running on GCE node
W20260620 09:21:31.886500 10776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:31.886608 10773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:31.886813 10768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:31.887010 10768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:31.888170 10768 hybrid_clock.cc:648] HybridClock initialized: now 1781947291888158 us; error 45 us; skew 500 ppm
Jun 20 09:21:31 dist-test-slave-414w krb5kdc[10393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947291, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:31.890746 10768 init.cc:377] Logged in from keytab as oryx/127.4.59.3@KRBTEST.COM (short username oryx)
I20260620 09:21:31.891214 10768 webserver.cc:460] Webserver started at http://127.4.59.3:42583/ using document root <none> and password file <none>
I20260620 09:21:31.891454 10768 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:31.891521 10768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:31.891677 10768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:31.892700 10768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "7e40b22cb65e462db3c82c933444b5fb"
format_stamp: "Formatted at 2026-06-20 09:21:31 on dist-test-slave-414w"
server_key: "6640a825bb84c927fef05975d2569a4b"
server_key_iv: "690e1af7570700985f636058e60d5785"
server_key_version: "encryptionkey@0"
I20260620 09:21:31.893059 10768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "7e40b22cb65e462db3c82c933444b5fb"
format_stamp: "Formatted at 2026-06-20 09:21:31 on dist-test-slave-414w"
server_key: "6640a825bb84c927fef05975d2569a4b"
server_key_iv: "690e1af7570700985f636058e60d5785"
server_key_version: "encryptionkey@0"
I20260620 09:21:31.894456 10768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:21:31.895231 10783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:31.895416 10768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 09:21:31.895512 10768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "7e40b22cb65e462db3c82c933444b5fb"
format_stamp: "Formatted at 2026-06-20 09:21:31 on dist-test-slave-414w"
server_key: "6640a825bb84c927fef05975d2569a4b"
server_key_iv: "690e1af7570700985f636058e60d5785"
server_key_version: "encryptionkey@0"
I20260620 09:21:31.895596 10768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:31.911388 10768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:31.913789 10768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:31.913965 10768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:31.914213 10768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:31.914520 10768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:31.914575 10768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:31.914618 10768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:31.914641 10768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:31.920557 10768 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:37999
I20260620 09:21:31.920595 10896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:37999 every 8 connection(s)
I20260620 09:21:31.920943 10768 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
Jun 20 09:21:31 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947291, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM
I20260620 09:21:31.930441 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:31.921708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:45521 (local address 127.4.59.62:46555)
0620 09:21:31.921754 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:31.921756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:31.921983 (+ 227us) server_negotiation.cc:407] Connection header received
0620 09:21:31.922435 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:31.922436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:31.922446 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:31.922506 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:31.923402 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:31.923964 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:31.925044 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:31.925405 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:31.925480 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:31.927574 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:31.927576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:31.927576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:31.927583 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:31.929051 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:31.929492 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:31.929493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:31.929493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:31.929544 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:31.929746 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:31.929747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:31.929747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:31.929860 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:31.929910 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:31.930231 (+ 321us) server_negotiation.cc:299] Negotiation successful
0620 09:21:31.930298 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 09:21:31.930516 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 10768
I20260620 09:21:31.930601 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:21:31.930606 10899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:31.921665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:45521)
0620 09:21:31.921823 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:31.921866 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:31.922352 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:31.922557 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:31.922560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:31.923108 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:31.923295 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:31.923300 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:31.924124 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:31.924125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:31.924950 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:31.924952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:31.925505 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:31.925506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:31.925606 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:31.926139 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:31.926147 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:31.927468 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:31.929138 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:31.929195 (+ 57us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:31.929197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:31.929425 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:31.929603 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:31.929603 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:31.929603 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:31.929693 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:31.929954 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:31.929959 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:31.930078 (+ 119us) client_negotiation.cc:769] Sending connection context
0620 09:21:31.930177 (+ 99us) client_negotiation.cc:240] Negotiation successful
0620 09:21:31.930358 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260620 09:21:31.930823 4332 external_mini_cluster.cc:1428] Setting key 4c6a820f91aee30dd4da735ff87cb061
I20260620 09:21:31.931039 10897 heartbeater.cc:347] Connected to a master server at 127.4.59.62:46555
I20260620 09:21:31.931149 10897 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:31.931352 10897 heartbeater.cc:511] Master 127.4.59.62:46555 requested a full tablet report, sending...
I20260620 09:21:31.931771 10438 ts_manager.cc:194] Registered new tserver with Master: 7e40b22cb65e462db3c82c933444b5fb (127.4.59.3:37999)
I20260620 09:21:31.932149 10438 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:45521
I20260620 09:21:31.942642 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.59.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.59.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.59.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.61.keytab.
Entry for principal oryx/127.4.59.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.61.keytab.
I20260620 09:21:31.967103 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.61:38809
--webserver_interface=127.4.59.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.61.keytab
--principal=oryx/127.4.59.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:46555,127.4.59.61:38809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:32.047792 10902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:32.048018 10902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:32.048043 10902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:32.049553 10902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:32.049611 10902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:32.049628 10902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:32.049640 10902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:32.049654 10902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:21:32.051218 10902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:46555,127.4.59.61:38809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.61.keytab
--principal=oryx/127.4.59.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.61:38809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.59.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
W20260620 09:21:32.051330 10902 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.4.59.62:46555,127.4.59.61:38809'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260620 09:21:32.051446 10902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:32.051620 10902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:32.056406 10910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:32.056421 10908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:32.056793 10902 server_base.cc:1034] running on GCE node
W20260620 09:21:32.056411 10907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:32.057123 10902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:32.057449 10902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:32.058632 10902 hybrid_clock.cc:648] HybridClock initialized: now 1781947292058626 us; error 37 us; skew 500 ppm
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:32.061317 10902 init.cc:377] Logged in from keytab as oryx/127.4.59.61@KRBTEST.COM (short username oryx)
I20260620 09:21:32.061761 10902 webserver.cc:460] Webserver started at http://127.4.59.61:40803/ using document root <none> and password file <none>
I20260620 09:21:32.061942 10902 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:32.061982 10902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:32.062072 10902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:32.063109 10902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/data/instance:
uuid: "849c62a684684348ba2a21bfa6a20637"
format_stamp: "Formatted at 2026-06-20 09:21:32 on dist-test-slave-414w"
server_key: "bf894f98113ab20e27500288ceb2c71a"
server_key_iv: "57b54ba6396f32373f44bed59e7b7dfc"
server_key_version: "encryptionkey@0"
I20260620 09:21:32.063411 10902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/wal/instance:
uuid: "849c62a684684348ba2a21bfa6a20637"
format_stamp: "Formatted at 2026-06-20 09:21:32 on dist-test-slave-414w"
server_key: "bf894f98113ab20e27500288ceb2c71a"
server_key_iv: "57b54ba6396f32373f44bed59e7b7dfc"
server_key_version: "encryptionkey@0"
I20260620 09:21:32.064731 10902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 09:21:32.065500 10917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.065685 10902 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.065758 10902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/wal
uuid: "849c62a684684348ba2a21bfa6a20637"
format_stamp: "Formatted at 2026-06-20 09:21:32 on dist-test-slave-414w"
server_key: "bf894f98113ab20e27500288ceb2c71a"
server_key_iv: "57b54ba6396f32373f44bed59e7b7dfc"
server_key_version: "encryptionkey@0"
I20260620 09:21:32.065806 10902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:32.092178 10902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:32.094172 10902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:32.094285 10902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:32.098290 10902 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.61:38809
I20260620 09:21:32.098377 10969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.61:38809 every 8 connection(s)
I20260620 09:21:32.098701 10902 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/data/info.pb
I20260620 09:21:32.099891 10970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:32.101454 10970 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } has no permanent_uuid. Determining permanent_uuid...
I20260620 09:21:32.101925 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 10902
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.61@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.59.61@KRBTEST.COM
I20260620 09:21:32.112077 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.102177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60982 (local address 127.4.59.62:46555)
0620 09:21:32.102260 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.102263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.102453 (+ 190us) server_negotiation.cc:407] Connection header received
0620 09:21:32.102956 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.102958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.102971 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.103030 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.103782 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.104636 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.105987 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.106320 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.106400 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.108784 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.108786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.108786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.108794 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.110497 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.110998 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.110999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.110999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.111064 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.111375 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.111376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.111376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.111534 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.111616 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.111904 (+ 288us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.111986 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 09:21:32.112263 10972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.102161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:60982)
0620 09:21:32.102313 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.102352 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.102847 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.103069 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.103072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.103447 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.103667 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.103673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.104759 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.104760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.105860 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.105863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.106409 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.106410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.106498 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.107053 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.107063 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.108678 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.110582 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.110586 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.110589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.110898 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.111182 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.111183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.111183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.111298 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.111608 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.111614 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.111753 (+ 139us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.111883 (+ 130us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.112034 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260620 09:21:32.112391 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.102472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44004 (local address 127.4.59.61:38809)
0620 09:21:32.102623 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.102628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.102713 (+ 85us) server_negotiation.cc:407] Connection header received
0620 09:21:32.102862 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.102868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.102889 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.103005 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.103735 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.104978 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.106056 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.106511 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.106562 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.109363 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.109369 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.109369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.109375 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.111067 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.111413 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.111415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.111416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.111473 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.111668 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.111669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.111669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.111812 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.111977 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.112056 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.112245 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":58,"threads_started":1}
I20260620 09:21:32.112960 10970 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.61@KRBTEST.COM for oryx/127.4.59.61@KRBTEST.COM
I20260620 09:21:32.118818 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.113265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44018 (local address 127.4.59.61:38809)
0620 09:21:32.113296 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.113298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.113305 (+ 7us) server_negotiation.cc:407] Connection header received
0620 09:21:32.113383 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.113384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.113393 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.113445 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.113691 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.114144 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.114794 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.114989 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.115070 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.116591 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.116592 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.116592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.116597 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.117869 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.118214 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.118215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.118216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.118263 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.118432 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.118432 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.118433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.118519 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.118567 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.118667 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.118761 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":18}
I20260620 09:21:32.118968 10972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.113241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:38809 (local address 127.0.0.1:44018)
0620 09:21:32.113268 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.113277 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.113315 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.113448 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.113448 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.113575 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.113617 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.113619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.114235 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.114236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.114706 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.114707 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.115070 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.115070 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.115128 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.115459 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.115462 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.116517 (+ 1055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.117971 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.117972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.117972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.118133 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.118327 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.118328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.118328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.118378 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.118572 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.118572 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.118603 (+ 31us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.118652 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.118719 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13}
I20260620 09:21:32.120213 10970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: Bootstrap starting.
I20260620 09:21:32.120967 10970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:32.121268 10970 log.cc:824] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:32.121897 10970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: No bootstrap required, opened a new log
I20260620 09:21:32.123344 10970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } }
I20260620 09:21:32.123474 10970 raft_consensus.cc:374] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:32.123497 10970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 849c62a684684348ba2a21bfa6a20637, State: Initialized, Role: FOLLOWER
I20260620 09:21:32.123616 10970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } }
I20260620 09:21:32.123855 10970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:32.123848 10976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } } }
I20260620 09:21:32.123986 10976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 09:21:32.125494 10987 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:21:32.125543 10987 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:21:32.127188 10902 master_runner.cc:428] Detected that this master 849c62a684684348ba2a21bfa6a20637 is joining an existing cluster
I20260620 09:21:32.127254 10902 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.59.61:38809
I20260620 09:21:32.134855 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.128221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60984 (local address 127.4.59.62:46555)
0620 09:21:32.128260 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.128262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.128269 (+ 7us) server_negotiation.cc:407] Connection header received
0620 09:21:32.128291 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.128292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.128303 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.128376 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.128700 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.129434 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.130462 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.130759 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.130844 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.132090 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.132092 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.132092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.132099 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.133677 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.134076 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.134077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.134078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.134134 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.134348 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.134349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.134349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.134466 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.134540 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.134698 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.134766 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:21:32.134881 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.128025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:60984)
0620 09:21:32.128169 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.128180 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.128227 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.128377 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.128378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.128546 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.128635 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.128637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.129521 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.129521 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.130375 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.130376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.130854 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.130855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.130912 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.131620 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.131624 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.132008 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.133775 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.133776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.133776 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.134001 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.134207 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.134208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.134208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.134266 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.134532 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.134532 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.134573 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.134671 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.134766 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260620 09:21:32.135840 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.128560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:38809 (local address 127.0.0.1:44028)
0620 09:21:32.129286 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.129296 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.129355 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.129499 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.129500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.129669 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.129728 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.129729 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.130517 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.130518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.131190 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.131192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.131546 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.131547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.131610 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.132311 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.132315 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.132650 (+ 335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.134750 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.134751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.134751 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.134954 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.135167 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.135168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.135168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.135234 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.135505 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.135506 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.135542 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.135640 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.135735 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":88,"threads_started":1}
I20260620 09:21:32.135962 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.128586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44028 (local address 127.4.59.61:38809)
0620 09:21:32.128630 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.128632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.129366 (+ 734us) server_negotiation.cc:407] Connection header received
0620 09:21:32.129408 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.129409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.129419 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.129486 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.129807 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.130458 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.131266 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.131465 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.131545 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.132741 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.132742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.132742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.132749 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.134663 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.135042 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.135042 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.135043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.135093 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.135301 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.135302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.135302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.135426 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.135512 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.135767 (+ 255us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.135826 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947291, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.62@KRBTEST.COM for oryx/127.4.59.61@KRBTEST.COM
I20260620 09:21:32.144519 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.137415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44040 (local address 127.4.59.61:38809)
0620 09:21:32.137445 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.137447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.137589 (+ 142us) server_negotiation.cc:407] Connection header received
0620 09:21:32.137649 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.137650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.137658 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.137721 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.138091 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.138785 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.139484 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.139768 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.139834 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.142058 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.142059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.142060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.142067 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.143434 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.143851 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.143852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.143852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.143897 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.144084 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.144084 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.144085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.144168 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.144214 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.144415 (+ 201us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.144469 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 09:21:32.144624 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.137369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:38809 (local address 127.0.0.1:44040)
0620 09:21:32.137486 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.137524 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.137594 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.137731 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.137735 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.137979 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.138019 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.138020 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.138871 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.138872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.139409 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.139410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.139836 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.139836 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.139912 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.140355 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.140360 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.141930 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.143515 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.143522 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.143523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.143773 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.143958 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.143958 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.143959 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.144023 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.144228 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.144234 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.144285 (+ 51us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.144371 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.144483 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":73,"threads_started":1}
I20260620 09:21:32.145195 10438 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.59.61:38809: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "849c62a684684348ba2a21bfa6a20637"
member_type: NON_VOTER
last_known_addr {
host: "127.4.59.61"
port: 38809
}
attrs {
promote: true
}
}
dest_uuid: "0990b69bade74124ab980ca951f7a9ad"
cas_config_opid_index: -1
I20260620 09:21:32.145491 10438 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: NON_VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: true } }
I20260620 09:21:32.145990 10996 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 849c62a684684348ba2a21bfa6a20637 (127.4.59.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: NON_VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: true } } }
I20260620 09:21:32.146174 10944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:32.146840 10996 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.59.61:38809
I20260620 09:21:32.146979 10996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0990b69bade74124ab980ca951f7a9ad" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: NON_VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: true } } }
I20260620 09:21:32.147046 10944 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Refusing update from remote peer 0990b69bade74124ab980ca951f7a9ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260620 09:21:32.147065 10996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:32.147245 10902 master.cc:537] Master@127.4.59.61:38809 shutting down...
I20260620 09:21:32.147387 10996 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: NON_VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 09:21:32.148022 10425 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260620 09:21:32.148429 10425 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad -> Peer 849c62a684684348ba2a21bfa6a20637 (127.4.59.61:38809): Couldn't send request to peer 849c62a684684348ba2a21bfa6a20637. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 09:21:32.150094 10902 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:32.150197 10902 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:32.150246 10902 tablet_replica.cc:330] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: stopping tablet replica
I20260620 09:21:32.162426 10902 master.cc:552] Master@127.4.59.61:38809 shutdown complete.
I20260620 09:21:32.162472 10902 master_runner.cc:305] Clearing existing system tablet
I20260620 09:21:32.162596 10902 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:32.162642 10902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:32.163381 10902 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.002s
I20260620 09:21:32.163916 11001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.164026 10902 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:32.164080 10902 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:21:32.164105 10902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/wal
uuid: "849c62a684684348ba2a21bfa6a20637"
format_stamp: "Formatted at 2026-06-20 09:21:32 on dist-test-slave-414w"
server_key: "bf894f98113ab20e27500288ceb2c71a"
server_key_iv: "57b54ba6396f32373f44bed59e7b7dfc"
server_key_version: "encryptionkey@0"
I20260620 09:21:32.164664 10902 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:32.166247 10902 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 09:21:32.166323 10902 log.cc:1196] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260620 09:21:32.166518 10902 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: Deleting consensus metadata
I20260620 09:21:32.166639 10902 master_runner.cc:315] Copying system tablet from 127.4.59.62:46555
I20260620 09:21:32.167248 10902 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: tablet copy: Beginning tablet copy session from remote peer at address 127.4.59.62:46555
I20260620 09:21:32.173565 11007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.167831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:60990)
0620 09:21:32.167949 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.167958 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.168033 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.168167 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.168168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.168328 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.168390 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.168392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.169036 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.169037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.169795 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.169797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.170172 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.170173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.170237 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.170702 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.170707 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.170980 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.172667 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.172668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.172668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.172825 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.173017 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.173018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.173018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.173065 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.173335 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.173336 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.173374 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.173428 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.173496 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260620 09:21:32.173633 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.167566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60990 (local address 127.4.59.62:46555)
0620 09:21:32.167605 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.167607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.168017 (+ 410us) server_negotiation.cc:407] Connection header received
0620 09:21:32.168092 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.168093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.168107 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.168162 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.168469 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.168949 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.169892 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.170090 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.170171 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.171083 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.171084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.171084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.171091 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.172581 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.172893 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.172894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.172895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.172955 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.173124 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.173124 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.173125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.173271 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.173341 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.173470 (+ 129us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.173544 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 09:21:32.174130 10458 tablet_copy_service.cc:140] P 0990b69bade74124ab980ca951f7a9ad: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 849c62a684684348ba2a21bfa6a20637 ({username='oryx', principal='oryx/127.4.59.61@KRBTEST.COM'} at 127.0.0.1:60990)
I20260620 09:21:32.174225 10458 tablet_copy_service.cc:161] P 0990b69bade74124ab980ca951f7a9ad: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 849c62a684684348ba2a21bfa6a20637 at {username='oryx', principal='oryx/127.4.59.61@KRBTEST.COM'} at 127.0.0.1:60990: session id = 849c62a684684348ba2a21bfa6a20637-00000000000000000000000000000000
I20260620 09:21:32.174985 10458 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 09:21:32.175566 10902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 09:21:32.176391 10902 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: tablet copy: Starting download of 0 data blocks...
I20260620 09:21:32.176558 10902 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: tablet copy: Starting download of 1 WAL segments...
I20260620 09:21:32.177567 10902 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 09:21:32.178298 10458 tablet_copy_service.cc:342] P 0990b69bade74124ab980ca951f7a9ad: Request end of tablet copy session 849c62a684684348ba2a21bfa6a20637-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.59.61@KRBTEST.COM'} at 127.0.0.1:60990
I20260620 09:21:32.178359 10458 tablet_copy_service.cc:433] P 0990b69bade74124ab980ca951f7a9ad: ending tablet copy session 849c62a684684348ba2a21bfa6a20637-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 849c62a684684348ba2a21bfa6a20637
I20260620 09:21:32.180660 10902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:32.181789 11008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:32.181890 11009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:32.182065 10902 server_base.cc:1034] running on GCE node
W20260620 09:21:32.182101 11011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:32.182277 10902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:32.182430 10902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:32.183594 10902 hybrid_clock.cc:648] HybridClock initialized: now 1781947292183584 us; error 24 us; skew 500 ppm
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:32.185014 10902 init.cc:377] Logged in from keytab as oryx/127.4.59.61@KRBTEST.COM (short username oryx)
I20260620 09:21:32.185312 10902 webserver.cc:460] Webserver started at http://127.4.59.61:33823/ using document root <none> and password file <none>
I20260620 09:21:32.185428 10902 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:32.185483 10902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:32.186061 10902 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:21:32.186553 11018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.186697 10902 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:21:32.186754 10902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/wal
uuid: "849c62a684684348ba2a21bfa6a20637"
format_stamp: "Formatted at 2026-06-20 09:21:32 on dist-test-slave-414w"
server_key: "bf894f98113ab20e27500288ceb2c71a"
server_key_iv: "57b54ba6396f32373f44bed59e7b7dfc"
server_key_version: "encryptionkey@0"
I20260620 09:21:32.186806 10902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:32.206504 10902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:32.208374 10902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:32.211618 10902 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.61:38809
I20260620 09:21:32.211796 11070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.61:38809 every 8 connection(s)
I20260620 09:21:32.211979 10902 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-1/data/info.pb
I20260620 09:21:32.212785 11071 sys_catalog.cc:263] Verifying existing consensus state
I20260620 09:21:32.213295 11071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: Bootstrap starting.
I20260620 09:21:32.215946 11071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 09:21:32.216197 11071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: Bootstrap complete.
I20260620 09:21:32.216408 11071 raft_consensus.cc:348] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: NON_VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: true } }
I20260620 09:21:32.216492 11071 raft_consensus.cc:729] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 849c62a684684348ba2a21bfa6a20637, State: Initialized, Role: LEARNER
I20260620 09:21:32.216528 11071 consensus_queue.cc:260] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: NON_VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: true } }
I20260620 09:21:32.216678 11071 sys_catalog.cc:564] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:32.218295 11075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: NON_VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: true } } }
I20260620 09:21:32.218410 11075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [sys.catalog]: This master's current role is: LEARNER
I20260620 09:21:32.219043 11086 catalog_manager.cc:1259] Loaded cluster ID: 3d88ecb57cd148b887151187edc1a264
I20260620 09:21:32.219084 11086 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: loading cluster ID for follower catalog manager: success
I20260620 09:21:32.220335 11086 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: acquiring CA information for follower catalog manager: success
I20260620 09:21:32.220657 11086 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 09:21:32.222476 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.214149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44052 (local address 127.4.59.61:38809)
0620 09:21:32.214307 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.214309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.214319 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:21:32.214343 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.214343 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.214362 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.214446 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.215089 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.216089 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.217114 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.217466 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.217536 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.218997 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.218999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.218999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.219006 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.221081 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.221520 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.221521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.221521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.221576 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.221788 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.221789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.221789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.221882 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.222154 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.222318 (+ 164us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.222398 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for oryx/127.4.59.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.59.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.59.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.60.keytab.
Entry for principal oryx/127.4.59.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.60.keytab.
I20260620 09:21:32.252367 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.60:44935
--webserver_interface=127.4.59.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.60.keytab
--principal=oryx/127.4.59.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:46555,127.4.59.61:38809,127.4.59.60:44935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:21:32.346422 11089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:32.346602 11089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:32.346624 11089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:32.347810 11089 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:21:32.347852 11089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:32.347872 11089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:32.347885 11089 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:21:32.347898 11089 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:21:32.349359 11089 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:46555,127.4.59.61:38809,127.4.59.60:44935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.60.keytab
--principal=oryx/127.4.59.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.60:44935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.59.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:32.349603 11089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:32.349824 11089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:32.354485 11095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:32.354493 11094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:32.354493 11097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:32.354550 11089 server_base.cc:1034] running on GCE node
I20260620 09:21:32.354890 11089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:32.355170 11089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:32.356307 11089 hybrid_clock.cc:648] HybridClock initialized: now 1781947292356287 us; error 38 us; skew 500 ppm
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:32.358681 11089 init.cc:377] Logged in from keytab as oryx/127.4.59.60@KRBTEST.COM (short username oryx)
I20260620 09:21:32.359143 11089 webserver.cc:460] Webserver started at http://127.4.59.60:35921/ using document root <none> and password file <none>
I20260620 09:21:32.359372 11089 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:32.359441 11089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:32.359589 11089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:21:32.360569 11089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/data/instance:
uuid: "b6318a47e1b447d3b85bf090d0a3f762"
format_stamp: "Formatted at 2026-06-20 09:21:32 on dist-test-slave-414w"
server_key: "7a5e18645571bb1f77e3c3c113c8cc17"
server_key_iv: "495a1c7c57f0d0f4c3a5f7dba272e28a"
server_key_version: "encryptionkey@0"
I20260620 09:21:32.360932 11089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/wal/instance:
uuid: "b6318a47e1b447d3b85bf090d0a3f762"
format_stamp: "Formatted at 2026-06-20 09:21:32 on dist-test-slave-414w"
server_key: "7a5e18645571bb1f77e3c3c113c8cc17"
server_key_iv: "495a1c7c57f0d0f4c3a5f7dba272e28a"
server_key_version: "encryptionkey@0"
I20260620 09:21:32.362262 11089 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:21:32.362993 11104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.363168 11089 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:32.363255 11089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/wal
uuid: "b6318a47e1b447d3b85bf090d0a3f762"
format_stamp: "Formatted at 2026-06-20 09:21:32 on dist-test-slave-414w"
server_key: "7a5e18645571bb1f77e3c3c113c8cc17"
server_key_iv: "495a1c7c57f0d0f4c3a5f7dba272e28a"
server_key_version: "encryptionkey@0"
I20260620 09:21:32.363338 11089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:32.378638 11089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:32.380484 11089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:32.380638 11089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:32.384656 11089 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.60:44935
I20260620 09:21:32.384686 11156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.60:44935 every 8 connection(s)
I20260620 09:21:32.385062 11089 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/data/info.pb
I20260620 09:21:32.386090 11157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:32.387705 11157 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } has no permanent_uuid. Determining permanent_uuid...
I20260620 09:21:32.388100 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 11089
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.60@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.59.60@KRBTEST.COM
I20260620 09:21:32.398614 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.388644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60992 (local address 127.4.59.62:46555)
0620 09:21:32.388693 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.388695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.388826 (+ 131us) server_negotiation.cc:407] Connection header received
0620 09:21:32.389516 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.389518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.389531 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.389608 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.390136 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.390935 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.392267 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.392562 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.392632 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.395188 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.395190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.395190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.395198 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.396752 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.397248 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.397249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.397250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.397323 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.397605 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.397606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.397606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.397798 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.397966 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.398430 (+ 464us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.398537 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 09:21:32.398788 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.388484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:60992)
0620 09:21:32.388677 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.388726 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.389407 (+ 681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.389632 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.389635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.389894 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.390042 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.390046 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.391036 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.391037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.392192 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.392195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.392639 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.392640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.392732 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.393497 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.393507 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.395098 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.396833 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.396840 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.396843 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.397170 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.397400 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.397401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.397402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.397530 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.397989 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.397993 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.398259 (+ 266us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.398359 (+ 100us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.398535 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260620 09:21:32.399350 11160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.388850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59132 (local address 127.4.59.60:44935)
0620 09:21:32.388981 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.388985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.389017 (+ 32us) server_negotiation.cc:407] Connection header received
0620 09:21:32.389415 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.389425 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.389446 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.389550 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.390206 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.391308 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.392262 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.392721 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.392768 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.395664 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.395671 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.395672 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.395679 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.397606 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.398191 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.398194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.398195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.398253 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.398475 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.398476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.398476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.398627 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.399145 (+ 518us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.399182 (+ 37us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.399238 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":83,"threads_started":1}
I20260620 09:21:32.399515 11157 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.60@KRBTEST.COM for oryx/127.4.59.61@KRBTEST.COM
I20260620 09:21:32.405856 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.399766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:38809 (local address 127.0.0.1:44066)
0620 09:21:32.399873 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.399884 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.399924 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.400045 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.400046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.400147 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.400190 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.400191 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.400779 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.400779 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.401431 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.401433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.401837 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.401838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.401887 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.402377 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.402382 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.403447 (+ 1065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.404911 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.404911 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.404912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.405076 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.405361 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.405362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.405362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.405429 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.405650 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.405651 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.405684 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.405734 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.405798 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260620 09:21:32.405952 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.399839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44066 (local address 127.4.59.61:38809)
0620 09:21:32.399879 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.399881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.399913 (+ 32us) server_negotiation.cc:407] Connection header received
0620 09:21:32.399978 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.399979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.399987 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.400043 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.400231 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.400688 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.401545 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.401755 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.401814 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.403517 (+ 1703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.403519 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.403519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.403525 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.404834 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.405225 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.405226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.405227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.405289 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.405492 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.405493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.405493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.405595 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.405649 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.405765 (+ 116us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.405833 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:21:32.406292 11157 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.60" port: 44935 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.60@KRBTEST.COM for oryx/127.4.59.60@KRBTEST.COM
I20260620 09:21:32.412000 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.406547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:44935 (local address 127.0.0.1:59146)
0620 09:21:32.406593 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.406603 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.406639 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.406738 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.406738 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.406838 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.406864 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.406865 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.407421 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.407422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.407864 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.407865 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.408239 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.408240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.408274 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.408675 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.408680 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.409888 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.411152 (+ 1264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.411153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.411153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.411304 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.411492 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.411493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.411493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.411569 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.411784 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.411785 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.411816 (+ 31us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.411865 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.411943 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 09:21:32.412088 11160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.406594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59146 (local address 127.4.59.60:44935)
0620 09:21:32.406645 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.406647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.406655 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:21:32.406675 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.406675 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.406683 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.406734 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.406918 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.407351 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.407965 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.408172 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.408234 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.409961 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.409962 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.409962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.409968 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.411082 (+ 1114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.411364 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.411365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.411365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.411427 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.411633 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.411634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.411634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.411732 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.411782 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.411883 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.411996 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":34}
I20260620 09:21:32.413355 11157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: Bootstrap starting.
I20260620 09:21:32.414157 11157 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:32.414424 11157 log.cc:824] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:32.415144 11157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: No bootstrap required, opened a new log
I20260620 09:21:32.416657 11157 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 44935 } }
I20260620 09:21:32.416785 11157 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:32.416810 11157 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6318a47e1b447d3b85bf090d0a3f762, State: Initialized, Role: FOLLOWER
I20260620 09:21:32.416955 11157 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 44935 } }
I20260620 09:21:32.417284 11162 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 44935 } } }
I20260620 09:21:32.417325 11157 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:32.417382 11162 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 09:21:32.419108 11173 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:21:32.419163 11173 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:21:32.421447 11089 master_runner.cc:428] Detected that this master b6318a47e1b447d3b85bf090d0a3f762 is joining an existing cluster
I20260620 09:21:32.421485 11089 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.59.60:44935
I20260620 09:21:32.430172 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.422732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44080 (local address 127.4.59.61:38809)
0620 09:21:32.422772 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.422774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.422999 (+ 225us) server_negotiation.cc:407] Connection header received
0620 09:21:32.423119 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.423120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.423130 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.423160 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.423564 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.424312 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.425569 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.425866 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.425908 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.427456 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.427457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.427457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.427464 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.428913 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.429371 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.429372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.429373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.429436 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.429650 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.429651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.429652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.429768 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.429845 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.430002 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.430076 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 09:21:32.430187 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.422813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:38809 (local address 127.0.0.1:44080)
0620 09:21:32.422947 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.422955 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.423047 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.423219 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.423219 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.423431 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.423482 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.423483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.424421 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.424422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.425467 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.425470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.426003 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.426005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.426072 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.426991 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.426997 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.427321 (+ 324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.429021 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.429022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.429022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.429286 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.429505 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.429506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.429507 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.429584 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.429835 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.429835 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.429880 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.429977 (+ 97us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.430078 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":29,"thread_start_us":78,"threads_started":1}
I20260620 09:21:32.449359 11181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.423060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:44935 (local address 127.0.0.1:59150)
0620 09:21:32.423160 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.423170 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.423250 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.423399 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.423400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.423687 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.423748 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.423750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.424626 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.424627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.425510 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.425512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.425998 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.425998 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.426055 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.445625 (+ 19570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.445631 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.446110 (+ 479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.447838 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.447839 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.447840 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.448398 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.448621 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.448622 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.448622 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.448674 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.448932 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.448933 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.448981 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.449075 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.449205 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":55,"threads_started":1}
I20260620 09:21:32.449611 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.422732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32772 (local address 127.4.59.62:46555)
0620 09:21:32.422854 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.422856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.422904 (+ 48us) server_negotiation.cc:407] Connection header received
0620 09:21:32.423014 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.423015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.423028 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.423060 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.423551 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.424322 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.425570 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.425866 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.425915 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.446310 (+ 20395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.446312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.446312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.446321 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.448019 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.448602 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.448603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.448604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.448665 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.449046 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.449048 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.449048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.449216 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.449281 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.449474 (+ 193us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.449554 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260620 09:21:32.449426 11160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.422997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59150 (local address 127.4.59.60:44935)
0620 09:21:32.423028 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.423030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.423218 (+ 188us) server_negotiation.cc:407] Connection header received
0620 09:21:32.423316 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.423317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.423324 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.423351 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.423847 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.424539 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.425600 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.425883 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.425921 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.446225 (+ 20304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.446226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.446227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.446234 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.447728 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.448484 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.448485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.448486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.448545 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.448739 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.448740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.448740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.448869 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.448921 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.449102 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.449274 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":17}
I20260620 09:21:32.449990 11179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.422696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:32772)
0620 09:21:32.422827 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.422839 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.422931 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.423124 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.423125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.423379 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.423455 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.423457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.424414 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.424415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.425468 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.425470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.426005 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.426005 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.426072 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.445791 (+ 19719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.445796 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.446126 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.448262 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.448263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.448264 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.448524 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.448939 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.448939 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.448940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.448979 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.449307 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.449308 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.449358 (+ 50us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.449762 (+ 404us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.449859 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":69,"thread_start_us":78,"threads_started":1}
I20260620 09:21:32.450853 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 10500
I20260620 09:21:32.457638 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:34235
--local_ip_for_outbound_sockets=127.4.59.1
--tserver_master_addrs=127.4.59.62:46555
--webserver_port=34041
--webserver_interface=127.4.59.1
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.1.keytab
--principal=oryx/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.59.62:46555,127.4.59.61:38809
--tserver_master_addrs=127.4.59.62:46555,127.4.59.61:38809,127.4.59.60:44935 with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947291, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.62@KRBTEST.COM for oryx/127.4.59.60@KRBTEST.COM
I20260620 09:21:32.463039 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.454560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:44935 (local address 127.0.0.1:59166)
0620 09:21:32.454609 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.454618 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.454657 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.456773 (+ 2116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.456774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.456930 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.456970 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.456972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.457695 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.457696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.458211 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.458212 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.458683 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.458684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.458755 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.459167 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.459172 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.460377 (+ 1205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.462012 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.462013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.462014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.462186 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.462411 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.462412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.462412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.462470 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.462716 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.462716 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.462761 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.462848 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.462949 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260620 09:21:32.463037 11160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.456581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59166 (local address 127.4.59.60:44935)
0620 09:21:32.456640 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.456642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.456649 (+ 7us) server_negotiation.cc:407] Connection header received
0620 09:21:32.456674 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.456675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.456683 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.456751 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.457055 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.457612 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.458307 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.458598 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.458685 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.460465 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.460467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.460467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.460474 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.461913 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.462272 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.462273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.462273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.462329 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.462539 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.462540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.462541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.462648 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.462722 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.462870 (+ 148us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.462944 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 09:21:32.463912 10438 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.59.60:44935: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762"
member_type: NON_VOTER
last_known_addr {
host: "127.4.59.60"
port: 44935
}
attrs {
promote: true
}
}
dest_uuid: "0990b69bade74124ab980ca951f7a9ad"
cas_config_opid_index: 5
I20260620 09:21:32.464169 10438 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: NON_VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: true } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } }
I20260620 09:21:32.464524 10997 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b6318a47e1b447d3b85bf090d0a3f762 (127.4.59.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: NON_VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: true } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } } }
I20260620 09:21:32.465408 10997 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.59.60:44935
I20260620 09:21:32.465490 10997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0990b69bade74124ab980ca951f7a9ad" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: NON_VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: true } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } } }
I20260620 09:21:32.465569 10997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:32.465858 11089 master.cc:537] Master@127.4.59.60:44935 shutting down...
I20260620 09:21:32.466166 11131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:32.467209 11131 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [term 1 FOLLOWER]: Refusing update from remote peer 0990b69bade74124ab980ca951f7a9ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 09:21:32.467485 10996 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 09:21:32.468206 10424 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad -> Peer b6318a47e1b447d3b85bf090d0a3f762 (127.4.59.60:44935): Couldn't send request to peer b6318a47e1b447d3b85bf090d0a3f762. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 09:21:32.469051 11089 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:32.469161 11089 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:32.469208 11089 tablet_replica.cc:330] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: stopping tablet replica
I20260620 09:21:32.471720 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.465663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44084 (local address 127.4.59.61:38809)
0620 09:21:32.465708 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.465711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.467883 (+ 2172us) server_negotiation.cc:407] Connection header received
0620 09:21:32.467943 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.467944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.467965 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.468019 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:32.468268 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.468981 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.470555 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.471181 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.471232 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.471305 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.471617 (+ 312us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.471659 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:21:32.471912 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.464617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:38809 (local address 127.0.0.1:44084)
0620 09:21:32.467772 (+ 3155us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.467783 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.467829 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.468138 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.468139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.468140 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:32.468190 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.468191 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.469076 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.469077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.470461 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.470465 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.471365 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.471365 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.471431 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.471450 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.471502 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.471841 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3140}
I20260620 09:21:32.472162 11045 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 LEARNER]: Refusing update from remote peer 0990b69bade74124ab980ca951f7a9ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 09:21:32.472342 10996 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: NON_VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260620 09:21:32.472373 11089 master.cc:552] Master@127.4.59.60:44935 shutdown complete.
I20260620 09:21:32.472414 11089 master_runner.cc:305] Clearing existing system tablet
I20260620 09:21:32.472546 11089 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:32.472590 11089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:32.472909 11045 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b6318a47e1b447d3b85bf090d0a3f762 (127.4.59.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: NON_VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: true } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } } }
I20260620 09:21:32.473938 11185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0990b69bade74124ab980ca951f7a9ad" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: NON_VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: true } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } } }
I20260620 09:21:32.474089 11185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [sys.catalog]: This master's current role is: LEARNER
I20260620 09:21:32.474130 10996 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad: attempting to promote NON_VOTER 849c62a684684348ba2a21bfa6a20637 to VOTER
I20260620 09:21:32.474174 11075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0990b69bade74124ab980ca951f7a9ad. Latest consensus state: current_term: 1 leader_uuid: "0990b69bade74124ab980ca951f7a9ad" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: NON_VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: true } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } } }
I20260620 09:21:32.474269 11075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [sys.catalog]: This master's current role is: LEARNER
I20260620 09:21:32.474318 10996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } }
I20260620 09:21:32.474751 11045 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 LEARNER]: Refusing update from remote peer 0990b69bade74124ab980ca951f7a9ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260620 09:21:32.474814 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.474697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:44935 (local address 127.0.0.1:59182)
0620 09:21:32.474753 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.474789 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260620 09:21:32.475019 11186 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260620 09:21:32.475174 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.475014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:44935 (local address 127.0.0.1:59194)
0620 09:21:32.475129 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.475151 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
W20260620 09:21:32.475219 10424 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad -> Peer b6318a47e1b447d3b85bf090d0a3f762 (127.4.59.60:44935): Couldn't send request to peer b6318a47e1b447d3b85bf090d0a3f762. Status: Network error: Client connection negotiation failed: client connection to 127.4.59.60:44935: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 09:21:32.475662 10997 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 849c62a684684348ba2a21bfa6a20637 (127.4.59.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } } }
I20260620 09:21:32.475792 11045 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 849c62a684684348ba2a21bfa6a20637 (127.4.59.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } } }
I20260620 09:21:32.476593 10997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0990b69bade74124ab980ca951f7a9ad" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } } }
I20260620 09:21:32.476629 10996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0990b69bade74124ab980ca951f7a9ad" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } } }
I20260620 09:21:32.476672 10997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:32.476679 10996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad [sys.catalog]: This master's current role is: LEADER
I20260620 09:21:32.476955 11075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0990b69bade74124ab980ca951f7a9ad" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } } }
I20260620 09:21:32.477051 11075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:21:32.477351 11089 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260620 09:21:32.477821 11189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.477937 11089 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:32.477984 11089 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.478012 11089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/wal
uuid: "b6318a47e1b447d3b85bf090d0a3f762"
format_stamp: "Formatted at 2026-06-20 09:21:32 on dist-test-slave-414w"
server_key: "7a5e18645571bb1f77e3c3c113c8cc17"
server_key_iv: "495a1c7c57f0d0f4c3a5f7dba272e28a"
server_key_version: "encryptionkey@0"
I20260620 09:21:32.478633 11089 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:32.479858 11089 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 09:21:32.479944 11089 log.cc:1196] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260620 09:21:32.480136 11089 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: Deleting consensus metadata
I20260620 09:21:32.480273 11089 master_runner.cc:315] Copying system tablet from 127.4.59.62:46555
I20260620 09:21:32.480886 11089 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: tablet copy: Beginning tablet copy session from remote peer at address 127.4.59.62:46555
I20260620 09:21:32.487109 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.481319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32784 (local address 127.4.59.62:46555)
0620 09:21:32.481360 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.481362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.481371 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:21:32.481435 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.481436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.481446 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.481500 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.481871 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.482345 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.483151 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.483357 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.483432 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.484408 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.484409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.484409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.484417 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.485868 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.486247 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.486248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.486249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.486305 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.486514 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.486515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.486516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.486631 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.486711 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.486938 (+ 227us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.487014 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 09:21:32.487177 11195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.481198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:32784)
0620 09:21:32.481304 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.481315 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.481374 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.481506 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.481507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.481727 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.481787 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.481789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.482430 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.482432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.483070 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.483072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.483443 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.483444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.483507 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.483989 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.483994 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.484295 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.485977 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.485978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.485978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.486160 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.486378 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.486380 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.486380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.486450 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.486724 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.486725 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.486768 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.486904 (+ 136us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.487053 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":70,"threads_started":1}
I20260620 09:21:32.487504 10458 tablet_copy_service.cc:140] P 0990b69bade74124ab980ca951f7a9ad: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b6318a47e1b447d3b85bf090d0a3f762 ({username='oryx', principal='oryx/127.4.59.60@KRBTEST.COM'} at 127.0.0.1:32784)
I20260620 09:21:32.487553 10458 tablet_copy_service.cc:161] P 0990b69bade74124ab980ca951f7a9ad: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b6318a47e1b447d3b85bf090d0a3f762 at {username='oryx', principal='oryx/127.4.59.60@KRBTEST.COM'} at 127.0.0.1:32784: session id = b6318a47e1b447d3b85bf090d0a3f762-00000000000000000000000000000000
I20260620 09:21:32.488170 10458 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 09:21:32.488675 11089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 09:21:32.489830 11089 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: tablet copy: Starting download of 0 data blocks...
I20260620 09:21:32.489996 11089 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: tablet copy: Starting download of 1 WAL segments...
I20260620 09:21:32.490653 11089 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 09:21:32.491353 10458 tablet_copy_service.cc:342] P 0990b69bade74124ab980ca951f7a9ad: Request end of tablet copy session b6318a47e1b447d3b85bf090d0a3f762-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.59.60@KRBTEST.COM'} at 127.0.0.1:32784
I20260620 09:21:32.491403 10458 tablet_copy_service.cc:433] P 0990b69bade74124ab980ca951f7a9ad: ending tablet copy session b6318a47e1b447d3b85bf090d0a3f762-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b6318a47e1b447d3b85bf090d0a3f762
I20260620 09:21:32.493899 11089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:32.495126 11196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:32.498325 11197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:32.498400 11089 server_base.cc:1034] running on GCE node
W20260620 09:21:32.501323 11199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:32.501497 11089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:32.501673 11089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:32.502797 11089 hybrid_clock.cc:648] HybridClock initialized: now 1781947292502788 us; error 29 us; skew 500 ppm
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:32.504169 11089 init.cc:377] Logged in from keytab as oryx/127.4.59.60@KRBTEST.COM (short username oryx)
I20260620 09:21:32.504468 11089 webserver.cc:460] Webserver started at http://127.4.59.60:43433/ using document root <none> and password file <none>
I20260620 09:21:32.504587 11089 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:32.504622 11089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:32.505242 11089 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:32.505779 11206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.505913 11089 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:32.505959 11089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/wal
uuid: "b6318a47e1b447d3b85bf090d0a3f762"
format_stamp: "Formatted at 2026-06-20 09:21:32 on dist-test-slave-414w"
server_key: "7a5e18645571bb1f77e3c3c113c8cc17"
server_key_iv: "495a1c7c57f0d0f4c3a5f7dba272e28a"
server_key_version: "encryptionkey@0"
I20260620 09:21:32.506016 11089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:32.522286 11089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:32.524048 11089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:32.527244 11089 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.60:44935
I20260620 09:21:32.527637 11089 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/master-2/data/info.pb
I20260620 09:21:32.529266 11258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.60:44935 every 8 connection(s)
I20260620 09:21:32.529850 11259 sys_catalog.cc:263] Verifying existing consensus state
I20260620 09:21:32.530262 11259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: Bootstrap starting.
I20260620 09:21:32.532562 11259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 09:21:32.532804 11259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: Bootstrap complete.
I20260620 09:21:32.532985 11259 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } }
I20260620 09:21:32.533059 11259 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b6318a47e1b447d3b85bf090d0a3f762, State: Initialized, Role: LEARNER
I20260620 09:21:32.533092 11259 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } }
I20260620 09:21:32.533272 11259 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:21:32.537292 11262 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } } }
I20260620 09:21:32.537405 11262 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762 [sys.catalog]: This master's current role is: LEARNER
I20260620 09:21:32.537797 11273 catalog_manager.cc:1259] Loaded cluster ID: 3d88ecb57cd148b887151187edc1a264
I20260620 09:21:32.537856 11273 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: loading cluster ID for follower catalog manager: success
I20260620 09:21:32.538650 11273 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: acquiring CA information for follower catalog manager: success
I20260620 09:21:32.538940 11273 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b6318a47e1b447d3b85bf090d0a3f762: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260620 09:21:32.576504 11182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:32.576792 11182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:32.576829 11182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:32.578540 11182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:32.578613 11182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:32.578655 11182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:21:32.580714 11182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.1.keytab
--principal=oryx/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:34235
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=34041
--tserver_master_addrs=127.4.59.62:46555,127.4.59.61:38809,127.4.59.60:44935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:32.580998 11182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:32.581279 11182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:32.586663 11281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:32.586668 11279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:32.586894 11182 server_base.cc:1034] running on GCE node
W20260620 09:21:32.586684 11278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:32.587210 11182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:32.587442 11182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:32.588606 11182 hybrid_clock.cc:648] HybridClock initialized: now 1781947292588587 us; error 27 us; skew 500 ppm
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:32.591454 11182 init.cc:377] Logged in from keytab as oryx/127.4.59.1@KRBTEST.COM (short username oryx)
I20260620 09:21:32.591868 11182 webserver.cc:460] Webserver started at http://127.4.59.1:34041/ using document root <none> and password file <none>
I20260620 09:21:32.592248 11182 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:32.592360 11182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:32.594197 11182 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:32.594862 11288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.595062 11182 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:32.595166 11182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "6ab00061f9c843e0bab91896d7649893"
format_stamp: "Formatted at 2026-06-20 09:21:31 on dist-test-slave-414w"
server_key: "27c9683403335b481028eac1f340ec5c"
server_key_iv: "6d0cf9f2b79a2ad6f42044703dbb7216"
server_key_version: "encryptionkey@0"
I20260620 09:21:32.595507 11182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:32.609318 11182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:32.611732 11182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:32.611888 11182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:32.612141 11182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:32.612460 11182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:32.612506 11182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.612542 11182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:32.612566 11182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.618769 11182 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:34235
I20260620 09:21:32.618891 11401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:34235 every 8 connection(s)
I20260620 09:21:32.619145 11182 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:21:32.623633 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 11182
I20260620 09:21:32.623772 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 10634
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.60@KRBTEST.COM
I20260620 09:21:32.629716 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:45543
--local_ip_for_outbound_sockets=127.4.59.2
--tserver_master_addrs=127.4.59.62:46555
--webserver_port=46597
--webserver_interface=127.4.59.2
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.2.keytab
--principal=oryx/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.59.62:46555,127.4.59.61:38809
--tserver_master_addrs=127.4.59.62:46555,127.4.59.61:38809,127.4.59.60:44935 with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:21:32.630355 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.619892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:56845 (local address 127.4.59.60:44935)
0620 09:21:32.620802 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.620804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.620812 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:21:32.620841 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.620842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.620852 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.620913 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.621670 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.622311 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.623649 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.624816 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.624886 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.627108 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.627110 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.627110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.627117 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.629017 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.629453 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.629454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.629455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.629512 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.629724 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.629725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.629725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.629854 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.629912 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.630127 (+ 215us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.630214 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":875,"thread_start_us":60,"threads_started":1}
I20260620 09:21:32.630388 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.619848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:38181)
0620 09:21:32.620095 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.620105 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.620639 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.620913 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.620918 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.621417 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.621579 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.621584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.622512 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.622512 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.623549 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.623551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.624030 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.624031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.624142 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.624626 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.624638 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.627013 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.628745 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.628751 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.628753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.628971 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.629245 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.629246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.629246 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.629384 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.629691 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.629698 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.629871 (+ 173us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.629988 (+ 117us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.630167 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":79,"spinlock_wait_cycles":384,"thread_start_us":93,"threads_started":1}
I20260620 09:21:32.630682 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.619848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:44935 (local address 127.4.59.1:56845)
0620 09:21:32.620011 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.620052 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.620626 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.620919 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.620920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.621398 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.621578 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.621582 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.622415 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.622416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.623553 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.623555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.624938 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.624939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.625004 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.625468 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.625474 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.627013 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.629111 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.629112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.629112 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.629366 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.629598 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.629599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.629599 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.629659 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.629939 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.629940 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.629982 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.630035 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.630523 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":460,"thread_start_us":88,"threads_started":1}
I20260620 09:21:32.630690 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.619860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:38181 (local address 127.4.59.62:46555)
0620 09:21:32.619913 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.619914 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.620164 (+ 250us) server_negotiation.cc:407] Connection header received
0620 09:21:32.620771 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.620772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.620798 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.620845 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.621687 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.622441 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.623651 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.623927 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.623977 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.627117 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.627118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.627119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.627125 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.628641 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.629064 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.629065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.629065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.629126 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.629455 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.629456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.629457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.629604 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.629665 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.630521 (+ 856us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.630610 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 09:21:32.631126 11402 heartbeater.cc:347] Connected to a master server at 127.4.59.60:44935
I20260620 09:21:32.631225 11402 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:32.631281 11403 heartbeater.cc:347] Connected to a master server at 127.4.59.62:46555
I20260620 09:21:32.631332 11403 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:32.631414 11402 heartbeater.cc:511] Master 127.4.59.60:44935 requested a full tablet report, sending...
I20260620 09:21:32.631414 11403 heartbeater.cc:511] Master 127.4.59.62:46555 requested a full tablet report, sending...
I20260620 09:21:32.631816 10438 ts_manager.cc:194] Re-registered known tserver with Master: 6ab00061f9c843e0bab91896d7649893 (127.4.59.1:34235)
I20260620 09:21:32.631886 11223 ts_manager.cc:194] Registered new tserver with Master: 6ab00061f9c843e0bab91896d7649893 (127.4.59.1:34235)
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.61@KRBTEST.COM
I20260620 09:21:32.632263 10438 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:38181
I20260620 09:21:32.635295 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.620372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:55787 (local address 127.4.59.61:38809)
0620 09:21:32.620435 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.620438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.620558 (+ 120us) server_negotiation.cc:407] Connection header received
0620 09:21:32.620724 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.620725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.620737 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.620804 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.621688 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.622436 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.623495 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.623791 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.623838 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.632702 (+ 8864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.632704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.632704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.632712 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.634155 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.634517 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.634518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.634519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.634573 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.634766 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.634767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.634768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.634907 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.634983 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.635137 (+ 154us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.635209 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 09:21:32.635454 11409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.620295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:38809 (local address 127.4.59.1:55787)
0620 09:21:32.620481 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.620497 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.620631 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.621269 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.621270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.621464 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.621579 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.621584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.622531 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.622532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.623375 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.623377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.629399 (+ 6022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.629399 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.629478 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.630102 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.630106 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.632569 (+ 2463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.634250 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.634251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.634252 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.634431 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.634640 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.634641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.634641 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.634696 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.634976 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.634977 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.635018 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.635101 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.635205 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":73,"thread_start_us":67,"threads_started":1}
I20260620 09:21:32.635859 11404 heartbeater.cc:347] Connected to a master server at 127.4.59.61:38809
I20260620 09:21:32.635923 11404 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:32.635993 11404 heartbeater.cc:511] Master 127.4.59.61:38809 requested a full tablet report, sending...
I20260620 09:21:32.636415 11035 ts_manager.cc:194] Registered new tserver with Master: 6ab00061f9c843e0bab91896d7649893 (127.4.59.1:34235)
W20260620 09:21:32.710703 11410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:32.710877 11410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:32.710901 11410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:32.711921 11410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:32.711988 11410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:32.712049 11410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:21:32.713514 11410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.2.keytab
--principal=oryx/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:45543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=46597
--tserver_master_addrs=127.4.59.62:46555,127.4.59.61:38809,127.4.59.60:44935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:32.713685 11410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:32.713842 11410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:32.718842 11415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:32.718947 11416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:32.718983 11410 server_base.cc:1034] running on GCE node
W20260620 09:21:32.718958 11418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:32.719449 11410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:32.719720 11410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:32.720885 11410 hybrid_clock.cc:648] HybridClock initialized: now 1781947292720870 us; error 45 us; skew 500 ppm
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:32.723472 11410 init.cc:377] Logged in from keytab as oryx/127.4.59.2@KRBTEST.COM (short username oryx)
I20260620 09:21:32.723939 11410 webserver.cc:460] Webserver started at http://127.4.59.2:46597/ using document root <none> and password file <none>
I20260620 09:21:32.724184 11410 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:32.724264 11410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:32.725757 11410 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:21:32.726344 11425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.726527 11410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:21:32.726634 11410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "e70728dae33f4444984b8f40f36d2d27"
format_stamp: "Formatted at 2026-06-20 09:21:31 on dist-test-slave-414w"
server_key: "a6612d99f94f9659d84717db811f52da"
server_key_iv: "5ad2df66e6bd7c3b0d7dae238a23d925"
server_key_version: "encryptionkey@0"
I20260620 09:21:32.726961 11410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:32.751431 11410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:32.753374 11410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:32.753526 11410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:32.753750 11410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:32.754053 11410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:32.754107 11410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.754149 11410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:32.754177 11410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.760182 11410 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:45543
I20260620 09:21:32.760226 11538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:45543 every 8 connection(s)
I20260620 09:21:32.760542 11410 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:21:32.765347 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 11410
I20260620 09:21:32.765502 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 10768
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.60@KRBTEST.COM
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.61@KRBTEST.COM
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM
I20260620 09:21:32.772112 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.761504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:58345 (local address 127.4.59.61:38809)
0620 09:21:32.761554 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.761556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.761569 (+ 13us) server_negotiation.cc:407] Connection header received
0620 09:21:32.762208 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.762209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.762222 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.762271 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.762980 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.763755 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.765012 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.765324 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.765393 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.768631 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.768632 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.768633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.768640 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.770502 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.770986 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.770987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.770987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.771043 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.771301 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.771301 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.771302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.771453 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.771496 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.771877 (+ 381us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.771966 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 09:21:32.772325 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.761266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:56249 (local address 127.4.59.60:44935)
0620 09:21:32.761327 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.761330 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.761722 (+ 392us) server_negotiation.cc:407] Connection header received
0620 09:21:32.762229 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.762231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.762247 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.762278 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.763018 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.763746 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.764825 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.765127 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.765213 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.769119 (+ 3906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.769120 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.769121 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.769128 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.771047 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.771462 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.771463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.771464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.771516 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.771720 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.771721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.771722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.771849 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.771890 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.772114 (+ 224us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.772207 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 09:21:32.772691 11544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.761270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:38809 (local address 127.4.59.2:58345)
0620 09:21:32.761441 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.761474 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.762101 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.762348 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.762352 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.762704 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.762884 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.762888 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.763949 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.763949 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.764895 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.764897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.765468 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.765469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.765551 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.766475 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.766480 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.768531 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.770592 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.770598 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.770600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.770901 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.771108 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.771109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.771109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.771234 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.771543 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.771549 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.771710 (+ 161us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.772229 (+ 519us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.772406 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":31,"spinlock_wait_cycles":7936,"thread_start_us":87,"threads_started":1}
I20260620 09:21:32.773044 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.761684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:48535 (local address 127.4.59.62:46555)
0620 09:21:32.761725 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.761727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.761861 (+ 134us) server_negotiation.cc:407] Connection header received
0620 09:21:32.762230 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.762231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.762244 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.762278 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.763102 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.763592 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.764983 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.765299 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.765361 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.769620 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.769621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.769622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.769629 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.771861 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.772217 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.772218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.772218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.772269 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.772468 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.772469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.772469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.772626 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.772664 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.772876 (+ 212us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.772953 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 09:21:32.773365 11545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.761628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:48535)
0620 09:21:32.761773 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.761789 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.762127 (+ 338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.762784 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.762785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.762926 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.762984 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.762986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.763693 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.763694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.764895 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.764897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.765410 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.765411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.765460 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.766311 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.766317 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.769536 (+ 3219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.771939 (+ 2403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.771939 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.771940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.772139 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.772336 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.772337 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.772337 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.772390 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.772708 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.772709 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.772758 (+ 49us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.773127 (+ 369us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.773259 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":83,"thread_start_us":71,"threads_started":1}
I20260620 09:21:32.773519 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.761269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:44935 (local address 127.4.59.2:56249)
0620 09:21:32.761552 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.761563 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.762133 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.762372 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.762372 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.762751 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.762886 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.762892 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.763826 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.763826 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.764715 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.764718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.765275 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.765276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.765419 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.766250 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.766258 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.769018 (+ 2760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.771151 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.771152 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.771153 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.771370 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.771591 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.771591 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.771592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.771651 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.771939 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.771939 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.771981 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.773276 (+ 1295us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.773356 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":120,"thread_start_us":89,"threads_started":1}
I20260620 09:21:32.773908 11539 heartbeater.cc:347] Connected to a master server at 127.4.59.60:44935
I20260620 09:21:32.774022 11539 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:32.774051 11540 heartbeater.cc:347] Connected to a master server at 127.4.59.62:46555
I20260620 09:21:32.774137 11540 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:32.774032 11541 heartbeater.cc:347] Connected to a master server at 127.4.59.61:38809
I20260620 09:21:32.774224 11539 heartbeater.cc:511] Master 127.4.59.60:44935 requested a full tablet report, sending...
I20260620 09:21:32.774250 11541 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:32.774252 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:37999
--local_ip_for_outbound_sockets=127.4.59.3
--tserver_master_addrs=127.4.59.62:46555
--webserver_port=42583
--webserver_interface=127.4.59.3
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.3.keytab
--principal=oryx/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.59.62:46555,127.4.59.61:38809
--tserver_master_addrs=127.4.59.62:46555,127.4.59.61:38809,127.4.59.60:44935 with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:21:32.774307 11541 heartbeater.cc:511] Master 127.4.59.61:38809 requested a full tablet report, sending...
I20260620 09:21:32.774227 11540 heartbeater.cc:511] Master 127.4.59.62:46555 requested a full tablet report, sending...
I20260620 09:21:32.774672 11035 ts_manager.cc:194] Registered new tserver with Master: e70728dae33f4444984b8f40f36d2d27 (127.4.59.2:45543)
I20260620 09:21:32.774946 10438 ts_manager.cc:194] Re-registered known tserver with Master: e70728dae33f4444984b8f40f36d2d27 (127.4.59.2:45543)
I20260620 09:21:32.775326 10438 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:48535
I20260620 09:21:32.775842 11223 ts_manager.cc:194] Registered new tserver with Master: e70728dae33f4444984b8f40f36d2d27 (127.4.59.2:45543)
W20260620 09:21:32.857328 11546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:21:32.857535 11546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:21:32.857559 11546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:21:32.858752 11546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:21:32.858810 11546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:21:32.858847 11546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:21:32.860472 11546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.3.keytab
--principal=oryx/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:37999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=42583
--tserver_master_addrs=127.4.59.62:46555,127.4.59.61:38809,127.4.59.60:44935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:21:32.860666 11546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:21:32.860998 11546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:21:32.866063 11552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:32.866115 11551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:21:32.866230 11554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:21:32.866775 11546 server_base.cc:1034] running on GCE node
I20260620 09:21:32.866991 11546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:21:32.867241 11546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:21:32.868418 11546 hybrid_clock.cc:648] HybridClock initialized: now 1781947292868418 us; error 43 us; skew 500 ppm
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:21:32.870956 11546 init.cc:377] Logged in from keytab as oryx/127.4.59.3@KRBTEST.COM (short username oryx)
I20260620 09:21:32.871377 11546 webserver.cc:460] Webserver started at http://127.4.59.3:42583/ using document root <none> and password file <none>
I20260620 09:21:32.871622 11546 fs_manager.cc:359] Metadata directory not provided
I20260620 09:21:32.871706 11546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:21:32.873083 11546 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:21:32.873735 11561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.873909 11546 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.874022 11546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "7e40b22cb65e462db3c82c933444b5fb"
format_stamp: "Formatted at 2026-06-20 09:21:31 on dist-test-slave-414w"
server_key: "6640a825bb84c927fef05975d2569a4b"
server_key_iv: "690e1af7570700985f636058e60d5785"
server_key_version: "encryptionkey@0"
I20260620 09:21:32.874372 11546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:21:32.893707 11546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:21:32.896036 11546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:21:32.896198 11546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:21:32.896436 11546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:21:32.896735 11546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:21:32.896788 11546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.896831 11546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:21:32.896857 11546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:21:32.902853 11546 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:37999
I20260620 09:21:32.902901 11674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:37999 every 8 connection(s)
I20260620 09:21:32.903226 11546 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:21:32.909608 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 11546
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.60@KRBTEST.COM
Jun 20 09:21:32 dist-test-slave-414w krb5kdc[10393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947292, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.61@KRBTEST.COM
I20260620 09:21:32.915186 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.904332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:39751 (local address 127.4.59.60:44935)
0620 09:21:32.904373 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.904374 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.904494 (+ 120us) server_negotiation.cc:407] Connection header received
0620 09:21:32.905282 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.905283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.905296 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.905329 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.906273 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.907056 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.908121 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.908394 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.908484 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.911391 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.911393 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.911393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.911400 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.913598 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.914060 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.914061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.914062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.914117 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.914358 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.914359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.914359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.914487 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.914544 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.915003 (+ 459us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.915089 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:21:32.916623 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.904257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:44207 (local address 127.4.59.62:46555)
0620 09:21:32.904299 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.904302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.904656 (+ 354us) server_negotiation.cc:407] Connection header received
0620 09:21:32.905266 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.905267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.905281 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.905321 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.906312 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.906929 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.907933 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.908185 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.908249 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.911911 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.911913 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.911913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.911920 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.914905 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.915721 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.915722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.915722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.915786 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.916020 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.916020 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.916021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.916141 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.916201 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.916444 (+ 243us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.916526 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 09:21:32.917460 11680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.904187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:44207)
0620 09:21:32.904612 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.904621 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.905175 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.905765 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.905766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.905949 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.906165 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.906169 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.907031 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.907032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.907827 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.907829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.908264 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.908265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.908392 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.908891 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.908899 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.911836 (+ 2937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.915293 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.915294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.915295 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.915513 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.915881 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.915882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.915882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.915948 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.916227 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.916227 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.916274 (+ 47us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.916337 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.917263 (+ 926us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":846,"thread_start_us":53,"threads_started":1}
I20260620 09:21:32.917655 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.903984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:38809 (local address 127.4.59.3:36389)
0620 09:21:32.904499 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.904508 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.905128 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.905425 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.905428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.905947 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.906165 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.906170 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.907130 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.907131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.908042 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.908044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.908566 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.908567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.908630 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.909268 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.909273 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.911982 (+ 2709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.916342 (+ 4360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.916342 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.916343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.916574 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.916920 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.916921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.916921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.917001 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.917307 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.917308 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.917350 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.917423 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.917525 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":74,"thread_start_us":71,"threads_started":1}
I20260620 09:21:32.917976 11676 heartbeater.cc:347] Connected to a master server at 127.4.59.62:46555
I20260620 09:21:32.918084 11676 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:32.918238 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.904047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:36389 (local address 127.4.59.61:38809)
0620 09:21:32.904093 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.904095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.904550 (+ 455us) server_negotiation.cc:407] Connection header received
0620 09:21:32.905283 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.905284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.905296 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.905329 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.906274 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.907032 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.908153 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.908446 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.908514 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.912067 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.912068 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.912068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.912075 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.916254 (+ 4179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.916690 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.916691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.916691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.916751 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.917083 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.917084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.917085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.917227 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.917283 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.918059 (+ 776us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.918152 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 09:21:32.918295 11676 heartbeater.cc:511] Master 127.4.59.62:46555 requested a full tablet report, sending...
I20260620 09:21:32.918664 10438 ts_manager.cc:194] Re-registered known tserver with Master: 7e40b22cb65e462db3c82c933444b5fb (127.4.59.3:37999)
I20260620 09:21:32.919155 10438 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:44207
I20260620 09:21:32.919449 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.910435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32800 (local address 127.4.59.62:46555)
0620 09:21:32.911786 (+ 1351us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.911789 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.911797 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:21:32.911819 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.911819 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.911832 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.911893 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.912644 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.913438 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.914514 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.914797 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.914851 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.916285 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.916286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.916286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.916292 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.918053 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.918455 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.918456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.918457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.918514 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.918747 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.918749 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.918749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.918885 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.918990 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.919279 (+ 289us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.919360 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1325,"thread_start_us":89,"threads_started":1}
I20260620 09:21:32.919920 11677 heartbeater.cc:347] Connected to a master server at 127.4.59.61:38809
I20260620 09:21:32.919983 11677 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:32.920477 11677 heartbeater.cc:511] Master 127.4.59.61:38809 requested a full tablet report, sending...
I20260620 09:21:32.920807 11035 ts_manager.cc:194] Registered new tserver with Master: 7e40b22cb65e462db3c82c933444b5fb (127.4.59.3:37999)
I20260620 09:21:32.921064 11681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.903912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:44935 (local address 127.4.59.3:39751)
0620 09:21:32.904368 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.904410 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.905128 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.905424 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.905428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.905930 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:21:32.906165 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.906170 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.907137 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.907138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.908043 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.908044 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.908485 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.908486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.908551 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.909067 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:21:32.909072 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:21:32.911276 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:21:32.913674 (+ 2398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.913682 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.913684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.913973 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.914180 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:21:32.914180 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:21:32.914181 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:21:32.914294 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:21:32.914568 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:21:32.914573 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:21:32.914721 (+ 148us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.914823 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.917036 (+ 2213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":33,"thread_start_us":76,"threads_started":1}
I20260620 09:21:32.921402 11675 heartbeater.cc:347] Connected to a master server at 127.4.59.60:44935
I20260620 09:21:32.921460 11675 heartbeater.cc:464] Registering TS with master...
I20260620 09:21:32.921519 11675 heartbeater.cc:511] Master 127.4.59.60:44935 requested a full tablet report, sending...
I20260620 09:21:32.921797 11223 ts_manager.cc:194] Registered new tserver with Master: 7e40b22cb65e462db3c82c933444b5fb (127.4.59.3:37999)
I20260620 09:21:32.923722 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.910619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44098 (local address 127.4.59.61:38809)
0620 09:21:32.911955 (+ 1336us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.911957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.911967 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:21:32.911987 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.911988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.911999 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.913237 (+ 1238us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.913276 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.914065 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.917623 (+ 3558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.917979 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.918771 (+ 792us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.920650 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.920652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.920652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.920659 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.922521 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.922957 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.922958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.922959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.923018 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.923260 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.923261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.923261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.923351 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.923463 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.923553 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.923623 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1317,"thread_start_us":54,"threads_started":1}
I20260620 09:21:32.924005 11691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.910863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59208 (local address 127.4.59.60:44935)
0620 09:21:32.911021 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.911023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.912209 (+ 1186us) server_negotiation.cc:407] Connection header received
0620 09:21:32.912229 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.912230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.912241 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.912295 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.914848 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.915617 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.918052 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.918354 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.918421 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.920858 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.920859 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.920859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.920866 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.922633 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.922998 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.922999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.922999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.923056 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.923279 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.923279 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.923280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.923401 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.923490 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.923840 (+ 350us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.923918 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":58,"threads_started":1}
I20260620 09:21:32.922302 10438 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:21:32.924542 10438 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:21:32.932025 11697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.927969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43702 (local address 127.4.59.3:37999)
0620 09:21:32.928826 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.928830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.928839 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:21:32.928867 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.928869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.928884 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.928975 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:32.929111 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.930134 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.931247 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.931734 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.931790 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.931836 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.931957 (+ 121us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.931981 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":89,"threads_started":1}
I20260620 09:21:32.932053 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.927888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:37999 (local address 127.0.0.1:43702)
0620 09:21:32.927936 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.927946 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.927990 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.928992 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.928993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.928996 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:32.929037 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.929038 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.930227 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.930227 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.931087 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.931089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.931818 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.931819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.931878 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.931882 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.931945 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.931986 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":1}
I20260620 09:21:32.932741 11609 tablet_service.cc:1467] Processing CreateTablet for tablet 2096b3f8cc34450f9b1d15a240a723bf (DEFAULT_TABLE table=test-table [id=0c6b57190f504a788a9f4836a895a6d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:32.933035 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.927969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:45543 (local address 127.0.0.1:46378)
0620 09:21:32.928057 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.928066 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.928092 (+ 26us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.928593 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.928594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.928596 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:32.928634 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.928635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.929972 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.929973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.931145 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.931147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.932783 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.932783 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.932844 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.932846 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.932916 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.932975 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260620 09:21:32.933097 11609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2096b3f8cc34450f9b1d15a240a723bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:32.933123 11696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.927988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:34235 (local address 127.0.0.1:46790)
0620 09:21:32.928433 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.928442 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.928492 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.928722 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.928723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.928725 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:32.928766 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.928768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.930135 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.930136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.931205 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.931207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.932886 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.932887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.932951 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.932956 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.933007 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.933051 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":64,"threads_started":1}
I20260620 09:21:32.933398 11698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.928184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46790 (local address 127.4.59.1:34235)
0620 09:21:32.928342 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.928347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.928567 (+ 220us) server_negotiation.cc:407] Connection header received
0620 09:21:32.928611 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.928613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.928630 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.928706 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:32.928853 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.930034 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.931288 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.931757 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.931857 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.931931 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.933284 (+ 1353us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.933324 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260620 09:21:32.933653 11699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.928179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46378 (local address 127.4.59.2:45543)
0620 09:21:32.928423 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.928427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.928436 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:21:32.928475 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.928477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.928494 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.928571 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:32.928717 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.929873 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.931966 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.932693 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.932771 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.932848 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.932981 (+ 133us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.933023 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":87,"threads_started":1}
I20260620 09:21:32.934049 11336 tablet_service.cc:1467] Processing CreateTablet for tablet 2096b3f8cc34450f9b1d15a240a723bf (DEFAULT_TABLE table=test-table [id=0c6b57190f504a788a9f4836a895a6d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:32.934360 11336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2096b3f8cc34450f9b1d15a240a723bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:32.934309 11473 tablet_service.cc:1467] Processing CreateTablet for tablet 2096b3f8cc34450f9b1d15a240a723bf (DEFAULT_TABLE table=test-table [id=0c6b57190f504a788a9f4836a895a6d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:21:32.934634 11473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2096b3f8cc34450f9b1d15a240a723bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:21:32.936652 11700 tablet_bootstrap.cc:492] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb: Bootstrap starting.
I20260620 09:21:32.937007 11701 tablet_bootstrap.cc:492] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893: Bootstrap starting.
I20260620 09:21:32.937409 11700 tablet_bootstrap.cc:654] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:32.937638 11701 tablet_bootstrap.cc:654] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:32.937728 11700 log.cc:824] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:32.937947 11701 log.cc:824] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:32.938282 11704 tablet_bootstrap.cc:492] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27: Bootstrap starting.
I20260620 09:21:32.938503 11700 tablet_bootstrap.cc:492] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb: No bootstrap required, opened a new log
I20260620 09:21:32.938589 11700 ts_tablet_manager.cc:1397] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:21:32.938755 11701 tablet_bootstrap.cc:492] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893: No bootstrap required, opened a new log
I20260620 09:21:32.938838 11701 ts_tablet_manager.cc:1397] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:21:32.938881 11704 tablet_bootstrap.cc:654] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27: Neither blocks nor log segments found. Creating new log.
I20260620 09:21:32.939177 11704 log.cc:824] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27: Log is configured to *not* fsync() on all Append() calls
I20260620 09:21:32.939952 11704 tablet_bootstrap.cc:492] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27: No bootstrap required, opened a new log
I20260620 09:21:32.940017 11704 ts_tablet_manager.cc:1397] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:21:32.940394 11700 raft_consensus.cc:348] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e40b22cb65e462db3c82c933444b5fb" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 37999 } } peers { permanent_uuid: "e70728dae33f4444984b8f40f36d2d27" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45543 } } peers { permanent_uuid: "6ab00061f9c843e0bab91896d7649893" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 34235 } }
I20260620 09:21:32.940725 11700 raft_consensus.cc:374] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:32.940775 11700 raft_consensus.cc:729] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e40b22cb65e462db3c82c933444b5fb, State: Initialized, Role: FOLLOWER
I20260620 09:21:32.940917 11700 consensus_queue.cc:260] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e40b22cb65e462db3c82c933444b5fb" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 37999 } } peers { permanent_uuid: "e70728dae33f4444984b8f40f36d2d27" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45543 } } peers { permanent_uuid: "6ab00061f9c843e0bab91896d7649893" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 34235 } }
I20260620 09:21:32.941224 11700 ts_tablet_manager.cc:1428] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:21:32.941354 11676 heartbeater.cc:503] Master 127.4.59.62:46555 was elected leader, sending a full tablet report...
I20260620 09:21:32.941541 11701 raft_consensus.cc:348] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e40b22cb65e462db3c82c933444b5fb" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 37999 } } peers { permanent_uuid: "e70728dae33f4444984b8f40f36d2d27" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45543 } } peers { permanent_uuid: "6ab00061f9c843e0bab91896d7649893" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 34235 } }
I20260620 09:21:32.941691 11701 raft_consensus.cc:374] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:32.942334 11701 raft_consensus.cc:729] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ab00061f9c843e0bab91896d7649893, State: Initialized, Role: FOLLOWER
I20260620 09:21:32.942409 11704 raft_consensus.cc:348] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e40b22cb65e462db3c82c933444b5fb" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 37999 } } peers { permanent_uuid: "e70728dae33f4444984b8f40f36d2d27" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45543 } } peers { permanent_uuid: "6ab00061f9c843e0bab91896d7649893" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 34235 } }
I20260620 09:21:32.942476 11701 consensus_queue.cc:260] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e40b22cb65e462db3c82c933444b5fb" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 37999 } } peers { permanent_uuid: "e70728dae33f4444984b8f40f36d2d27" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45543 } } peers { permanent_uuid: "6ab00061f9c843e0bab91896d7649893" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 34235 } }
I20260620 09:21:32.942552 11704 raft_consensus.cc:374] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:21:32.942590 11704 raft_consensus.cc:729] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70728dae33f4444984b8f40f36d2d27, State: Initialized, Role: FOLLOWER
I20260620 09:21:32.942716 11701 ts_tablet_manager.cc:1428] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260620 09:21:32.942703 11704 consensus_queue.cc:260] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e40b22cb65e462db3c82c933444b5fb" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 37999 } } peers { permanent_uuid: "e70728dae33f4444984b8f40f36d2d27" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45543 } } peers { permanent_uuid: "6ab00061f9c843e0bab91896d7649893" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 34235 } }
I20260620 09:21:32.942742 11403 heartbeater.cc:503] Master 127.4.59.62:46555 was elected leader, sending a full tablet report...
I20260620 09:21:32.943060 11704 ts_tablet_manager.cc:1428] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:21:32.943302 11540 heartbeater.cc:503] Master 127.4.59.62:46555 was elected leader, sending a full tablet report...
I20260620 09:21:32.954738 11706 raft_consensus.cc:482] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:21:32.954809 11706 raft_consensus.cc:504] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e40b22cb65e462db3c82c933444b5fb" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 37999 } } peers { permanent_uuid: "e70728dae33f4444984b8f40f36d2d27" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45543 } } peers { permanent_uuid: "6ab00061f9c843e0bab91896d7649893" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 34235 } }
I20260620 09:21:32.955235 11706 leader_election.cc:288] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e70728dae33f4444984b8f40f36d2d27 (127.4.59.2:45543), 6ab00061f9c843e0bab91896d7649893 (127.4.59.1:34235)
I20260620 09:21:32.957911 11681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.955370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:45543 (local address 127.4.59.3:52415)
0620 09:21:32.955419 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.955430 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.955469 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.955583 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.955584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.955585 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:32.955646 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.955647 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.956299 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.956301 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.957093 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.957095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.957660 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.957662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.957733 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.957735 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.957811 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.957852 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 09:21:32.957909 11699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.955445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:52415 (local address 127.4.59.2:45543)
0620 09:21:32.955481 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.955483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.955492 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:21:32.955516 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.955517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.955527 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.955572 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:32.955717 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.956219 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.957216 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.957571 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.957659 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.957668 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.957816 (+ 148us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.957848 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:21:32.958079 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.955403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:34235 (local address 127.4.59.3:52431)
0620 09:21:32.955450 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:32.955460 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:21:32.955500 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:21:32.955650 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:21:32.955651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:21:32.955652 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:21:32.955723 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.955724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.956523 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.956524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.957297 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:21:32.957299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.957813 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.957814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:21:32.957887 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.957889 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:21:32.957968 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 09:21:32.958006 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 09:21:32.958070 11698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.955459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:52431 (local address 127.4.59.1:34235)
0620 09:21:32.955493 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.955495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.955503 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:21:32.955555 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.955556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.955565 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.955645 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:32.955807 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.956428 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.957378 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.957722 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.957812 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.957820 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.957981 (+ 161us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.958015 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 09:21:32.958338 11356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2096b3f8cc34450f9b1d15a240a723bf" candidate_uuid: "7e40b22cb65e462db3c82c933444b5fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ab00061f9c843e0bab91896d7649893" is_pre_election: true
I20260620 09:21:32.958415 11493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2096b3f8cc34450f9b1d15a240a723bf" candidate_uuid: "7e40b22cb65e462db3c82c933444b5fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70728dae33f4444984b8f40f36d2d27" is_pre_election: true
I20260620 09:21:32.958490 11356 raft_consensus.cc:2454] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e40b22cb65e462db3c82c933444b5fb in term 0.
I20260620 09:21:32.958508 11493 raft_consensus.cc:2454] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e40b22cb65e462db3c82c933444b5fb in term 0.
I20260620 09:21:32.958698 11562 leader_election.cc:302] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e40b22cb65e462db3c82c933444b5fb, e70728dae33f4444984b8f40f36d2d27; no voters:
I20260620 09:21:32.958832 11706 raft_consensus.cc:2781] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:21:32.958884 11706 raft_consensus.cc:482] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:21:32.958925 11706 raft_consensus.cc:3037] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:32.959565 11706 raft_consensus.cc:504] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e40b22cb65e462db3c82c933444b5fb" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 37999 } } peers { permanent_uuid: "e70728dae33f4444984b8f40f36d2d27" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45543 } } peers { permanent_uuid: "6ab00061f9c843e0bab91896d7649893" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 34235 } }
I20260620 09:21:32.959708 11706 leader_election.cc:288] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [CANDIDATE]: Term 1 election: Requested vote from peers e70728dae33f4444984b8f40f36d2d27 (127.4.59.2:45543), 6ab00061f9c843e0bab91896d7649893 (127.4.59.1:34235)
I20260620 09:21:32.959849 11493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2096b3f8cc34450f9b1d15a240a723bf" candidate_uuid: "7e40b22cb65e462db3c82c933444b5fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70728dae33f4444984b8f40f36d2d27"
I20260620 09:21:32.959861 11356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2096b3f8cc34450f9b1d15a240a723bf" candidate_uuid: "7e40b22cb65e462db3c82c933444b5fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ab00061f9c843e0bab91896d7649893"
I20260620 09:21:32.959920 11356 raft_consensus.cc:3037] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:32.959920 11493 raft_consensus.cc:3037] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:21:32.960669 11356 raft_consensus.cc:2454] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e40b22cb65e462db3c82c933444b5fb in term 1.
I20260620 09:21:32.960669 11493 raft_consensus.cc:2454] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e40b22cb65e462db3c82c933444b5fb in term 1.
I20260620 09:21:32.960822 11562 leader_election.cc:302] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e40b22cb65e462db3c82c933444b5fb, e70728dae33f4444984b8f40f36d2d27; no voters:
I20260620 09:21:32.960923 11706 raft_consensus.cc:2781] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:21:32.961077 11706 raft_consensus.cc:686] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [term 1 LEADER]: Becoming Leader. State: Replica: 7e40b22cb65e462db3c82c933444b5fb, State: Running, Role: LEADER
I20260620 09:21:32.961215 11706 consensus_queue.cc:237] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e40b22cb65e462db3c82c933444b5fb" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 37999 } } peers { permanent_uuid: "e70728dae33f4444984b8f40f36d2d27" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45543 } } peers { permanent_uuid: "6ab00061f9c843e0bab91896d7649893" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 34235 } }
I20260620 09:21:32.961918 10438 catalog_manager.cc:5495] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e40b22cb65e462db3c82c933444b5fb (127.4.59.3). New cstate: current_term: 1 leader_uuid: "7e40b22cb65e462db3c82c933444b5fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e40b22cb65e462db3c82c933444b5fb" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 37999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e70728dae33f4444984b8f40f36d2d27" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ab00061f9c843e0bab91896d7649893" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260620 09:21:32.968580 11697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.966608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43714 (local address 127.4.59.3:37999)
0620 09:21:32.966645 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.966647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.966662 (+ 15us) server_negotiation.cc:407] Connection header received
0620 09:21:32.966687 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.966687 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.966700 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.966757 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:21:32.966866 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.967289 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.967972 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.968141 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.968194 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.968336 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:21:32.968413 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:21:32.968482 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.968522 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.968537 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:21:32.970120 11493 raft_consensus.cc:1261] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27 [term 1 FOLLOWER]: Refusing update from remote peer 7e40b22cb65e462db3c82c933444b5fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:32.970120 11356 raft_consensus.cc:1261] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893 [term 1 FOLLOWER]: Refusing update from remote peer 7e40b22cb65e462db3c82c933444b5fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:21:32.970422 11706 consensus_queue.cc:1035] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ab00061f9c843e0bab91896d7649893" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:32.970551 11709 consensus_queue.cc:1035] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e70728dae33f4444984b8f40f36d2d27" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:21:32.973922 11714 mvcc.cc:205] Tried to move back new op lower bound from 7298856112003391488 to 7298856111969931264. Current Snapshot: MvccSnapshot[applied={T|T < 7298856112003391488}]
I20260620 09:21:32.974022 11713 mvcc.cc:205] Tried to move back new op lower bound from 7298856112003391488 to 7298856111969931264. Current Snapshot: MvccSnapshot[applied={T|T < 7298856112003391488 or (T in {7298856112003391488})}]
I20260620 09:21:32.975888 10437 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:32.975997 10437 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:21:32.977720 10437 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0990b69bade74124ab980ca951f7a9ad: Sending DeleteTablet for 3 replicas of tablet 2096b3f8cc34450f9b1d15a240a723bf
I20260620 09:21:32.978036 11473 tablet_service.cc:1514] Processing DeleteTablet for tablet 2096b3f8cc34450f9b1d15a240a723bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:32 UTC) from {username='oryx', principal='oryx/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:46378
I20260620 09:21:32.978255 11721 tablet_replica.cc:330] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27: stopping tablet replica
I20260620 09:21:32.978348 11721 raft_consensus.cc:2229] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:32.978331 11609 tablet_service.cc:1514] Processing DeleteTablet for tablet 2096b3f8cc34450f9b1d15a240a723bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:32 UTC) from {username='oryx', principal='oryx/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:43702
I20260620 09:21:32.978456 11721 raft_consensus.cc:2258] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:32.978519 11722 tablet_replica.cc:330] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb: stopping tablet replica
I20260620 09:21:32.978578 11336 tablet_service.cc:1514] Processing DeleteTablet for tablet 2096b3f8cc34450f9b1d15a240a723bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:21:32 UTC) from {username='oryx', principal='oryx/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:46790
I20260620 09:21:32.978673 11722 raft_consensus.cc:2229] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:21:32.978777 11722 raft_consensus.cc:2258] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:32.978850 11721 ts_tablet_manager.cc:1905] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:32.979174 11722 ts_tablet_manager.cc:1905] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:32.980365 11721 ts_tablet_manager.cc:1918] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:21:32.980424 11721 log.cc:1196] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/2096b3f8cc34450f9b1d15a240a723bf
I20260620 09:21:32.981004 11721 ts_tablet_manager.cc:1939] T 2096b3f8cc34450f9b1d15a240a723bf P e70728dae33f4444984b8f40f36d2d27: Deleting consensus metadata
I20260620 09:21:32.981027 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 10406
I20260620 09:21:32.981707 11723 tablet_replica.cc:330] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893: stopping tablet replica
I20260620 09:21:32.981829 11723 raft_consensus.cc:2229] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:21:32.981943 11723 raft_consensus.cc:2258] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:21:32.982334 11723 ts_tablet_manager.cc:1905] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:21:32.984339 11723 ts_tablet_manager.cc:1918] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:21:32.984475 11723 log.cc:1196] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-0/wal/wals/2096b3f8cc34450f9b1d15a240a723bf
I20260620 09:21:32.984772 11723 ts_tablet_manager.cc:1939] T 2096b3f8cc34450f9b1d15a240a723bf P 6ab00061f9c843e0bab91896d7649893: Deleting consensus metadata
W20260620 09:21:32.987445 11428 connection.cc:535] server connection from 127.0.0.1:46378 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 09:21:32.987458 11290 connection.cc:535] server connection from 127.0.0.1:46790 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 09:21:32.987530 11691 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:21:32.977244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59220 (local address 127.4.59.60:44935)
0620 09:21:32.977287 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.977289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.977369 (+ 80us) server_negotiation.cc:407] Connection header received
0620 09:21:32.977454 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.977454 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.977478 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.977533 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:21:32.977685 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.978437 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.980585 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.981243 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.981307 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.981380 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.987455 (+ 6075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59220: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 09:21:32.988821 11722 ts_tablet_manager.cc:1918] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:21:32.988914 11722 log.cc:1196] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/2096b3f8cc34450f9b1d15a240a723bf
I20260620 09:21:32.989213 11722 ts_tablet_manager.cc:1939] T 2096b3f8cc34450f9b1d15a240a723bf P 7e40b22cb65e462db3c82c933444b5fb: Deleting consensus metadata
I20260620 09:21:32.989482 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.980489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59228 (local address 127.4.59.60:44935)
0620 09:21:32.980530 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.980532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.981599 (+ 1067us) server_negotiation.cc:407] Connection header received
0620 09:21:32.981661 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.981662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.981672 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.981719 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.982504 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.983201 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.984340 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.984640 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.984683 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.986378 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.986379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.986380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.986387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.988234 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.988640 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.988641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.988642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.988695 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.988894 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.988895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.988895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.989012 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.989086 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.989313 (+ 227us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.989385 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 09:21:32.989557 11562 connection.cc:441] server connection from 127.0.0.1:43702 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43702 (request call id 1) could send its response
I20260620 09:21:32.996953 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.990004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59230 (local address 127.4.59.60:44935)
0620 09:21:32.990040 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.990041 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.990361 (+ 320us) server_negotiation.cc:407] Connection header received
0620 09:21:32.990449 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.990450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.990459 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.990496 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.991197 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.991869 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.992598 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.992801 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.992887 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.994540 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.994541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.994541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.994549 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.995754 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.996149 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.996151 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.996151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.996208 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.996445 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.996446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.996446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.996569 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.996662 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.996798 (+ 136us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.996865 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:21:32.998000 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:32.989450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44114 (local address 127.4.59.61:38809)
0620 09:21:32.989491 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 09:21:32.989493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:21:32.989969 (+ 476us) server_negotiation.cc:407] Connection header received
0620 09:21:32.990072 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:21:32.990073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:21:32.990086 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:21:32.990115 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:21:32.991060 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.991847 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.992962 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:21:32.993298 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:21:32.993362 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:21:32.994622 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:21:32.994623 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:21:32.994624 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:21:32.994631 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:21:32.996809 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.997233 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.997234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.997234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.997290 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:21:32.997503 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:21:32.997504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:21:32.997504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:21:32.997612 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:21:32.997709 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 09:21:32.997814 (+ 105us) server_negotiation.cc:299] Negotiation successful
0620 09:21:32.997890 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:21:33.974387 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:33.974071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:33503)
0620 09:21:33.974271 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:33.974341 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":45,"threads_started":1}
I20260620 09:21:33.974571 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:33.973896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:46607)
0620 09:21:33.974039 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:33.974124 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260620 09:21:33.974669 11733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:33.974058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:60285)
0620 09:21:33.974434 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:33.974597 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":53,"threads_started":1}
W20260620 09:21:33.974803 11403 heartbeater.cc:644] Failed to heartbeat to 127.4.59.62:46555 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
W20260620 09:21:33.974838 11676 heartbeater.cc:644] Failed to heartbeat to 127.4.59.62:46555 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
W20260620 09:21:33.974843 11540 heartbeater.cc:644] Failed to heartbeat to 127.4.59.62:46555 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
I20260620 09:21:34.478051 11735 raft_consensus.cc:482] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0990b69bade74124ab980ca951f7a9ad)
I20260620 09:21:34.478227 11735 raft_consensus.cc:504] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } }
I20260620 09:21:34.478477 11735 leader_election.cc:288] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0990b69bade74124ab980ca951f7a9ad (127.4.59.62:46555)
I20260620 09:21:34.478819 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:34.478629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:33308)
0620 09:21:34.478731 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:34.478782 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":58,"threads_started":1}
W20260620 09:21:34.478961 11019 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
I20260620 09:21:34.479341 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:34.479212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:33316)
0620 09:21:34.479285 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:34.479318 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260620 09:21:34.479441 11019 leader_election.cc:334] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0990b69bade74124ab980ca951f7a9ad (127.4.59.62:46555): Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
I20260620 09:21:34.479518 11019 leader_election.cc:302] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 849c62a684684348ba2a21bfa6a20637; no voters: 0990b69bade74124ab980ca951f7a9ad
I20260620 09:21:34.479652 11735 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 09:21:34.975448 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:34.975177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:53579)
0620 09:21:34.975347 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:34.975391 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":55,"threads_started":1}
I20260620 09:21:34.975564 11739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:34.975177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:34253)
0620 09:21:34.975428 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:34.975519 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":58,"threads_started":1}
I20260620 09:21:34.975710 11738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:34.975153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:40539)
0620 09:21:34.975621 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:34.975659 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":63,"threads_started":1}
I20260620 09:21:35.976661 11743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:35.976320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:43163)
0620 09:21:35.976545 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:35.976597 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":52,"threads_started":1}
I20260620 09:21:35.976687 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:35.976204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:38317)
0620 09:21:35.976596 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:35.976642 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":62,"threads_started":1}
I20260620 09:21:35.976687 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:35.976200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:51005)
0620 09:21:35.976611 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:35.976643 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":70,"threads_started":1}
I20260620 09:21:36.139240 11744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0990b69bade74124ab980ca951f7a9ad)
I20260620 09:21:36.139335 11744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } }
I20260620 09:21:36.139508 11744 leader_election.cc:288] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0990b69bade74124ab980ca951f7a9ad (127.4.59.62:46555)
I20260620 09:21:36.139832 11745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:36.139655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:33530)
0620 09:21:36.139756 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:36.139797 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":47,"threads_started":1}
I20260620 09:21:36.140262 11745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:36.140164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:33536)
0620 09:21:36.140213 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:36.140244 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 09:21:36.140336 11019 leader_election.cc:334] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0990b69bade74124ab980ca951f7a9ad (127.4.59.62:46555): Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
I20260620 09:21:36.140380 11019 leader_election.cc:302] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 849c62a684684348ba2a21bfa6a20637; no voters: 0990b69bade74124ab980ca951f7a9ad
I20260620 09:21:36.140443 11744 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 09:21:36.977347 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:36.977091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:35525)
0620 09:21:36.977250 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:36.977290 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":96,"threads_started":1}
I20260620 09:21:36.977439 11747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:36.977063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:55983)
0620 09:21:36.977368 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:36.977400 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":65,"threads_started":1}
I20260620 09:21:36.977576 11749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:36.977281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:45309)
0620 09:21:36.977482 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:36.977520 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":67,"threads_started":1}
I20260620 09:21:37.978291 11752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:37.978099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:48549)
0620 09:21:37.978203 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:37.978239 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":43,"threads_started":1}
I20260620 09:21:37.978291 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:37.978078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:57765)
0620 09:21:37.978203 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:37.978240 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260620 09:21:37.978384 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:37.978075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:45053)
0620 09:21:37.978292 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:37.978332 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":62,"threads_started":1}
I20260620 09:21:38.954344 11753 raft_consensus.cc:482] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0990b69bade74124ab980ca951f7a9ad)
I20260620 09:21:38.954468 11753 raft_consensus.cc:504] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } }
I20260620 09:21:38.954648 11753 leader_election.cc:288] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0990b69bade74124ab980ca951f7a9ad (127.4.59.62:46555)
I20260620 09:21:38.955034 11754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:38.954854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:33824)
0620 09:21:38.954963 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:38.954997 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":57,"threads_started":1}
I20260620 09:21:38.955430 11754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:38.955344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:33834)
0620 09:21:38.955389 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:38.955414 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 09:21:38.955504 11019 leader_election.cc:334] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0990b69bade74124ab980ca951f7a9ad (127.4.59.62:46555): Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
I20260620 09:21:38.955581 11019 leader_election.cc:302] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 849c62a684684348ba2a21bfa6a20637; no voters: 0990b69bade74124ab980ca951f7a9ad
I20260620 09:21:38.955672 11753 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 09:21:38.978834 11756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:38.978632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:43927)
0620 09:21:38.978742 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:38.978776 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":55,"threads_started":1}
I20260620 09:21:38.978962 11757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:38.978753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:46765)
0620 09:21:38.978878 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:38.978914 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":83,"threads_started":1}
I20260620 09:21:38.979156 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:38.978760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:45775)
0620 09:21:38.979078 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:38.979112 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":59,"threads_started":1}
I20260620 09:21:39.979667 11760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:39.979459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:42983)
0620 09:21:39.979580 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:39.979616 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260620 09:21:39.979676 11759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:39.979459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:33283)
0620 09:21:39.979582 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:39.979616 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260620 09:21:39.979969 11761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:39.979507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:58077)
0620 09:21:39.979739 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:39.979786 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":54,"threads_started":1}
I20260620 09:21:40.980401 11762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:40.980032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:36175)
0620 09:21:40.980293 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:40.980342 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":64,"threads_started":1}
I20260620 09:21:40.980535 11764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:40.980331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:57995)
0620 09:21:40.980450 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:40.980486 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":43,"threads_started":1}
I20260620 09:21:40.980582 11763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:40.980197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:37485)
0620 09:21:40.980512 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:40.980540 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":42,"threads_started":1}
I20260620 09:21:41.981276 11767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:41.981040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:44995)
0620 09:21:41.981185 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:41.981222 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":58,"threads_started":1}
I20260620 09:21:41.981276 11765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:41.981029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:39879)
0620 09:21:41.981177 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:41.981220 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260620 09:21:41.981494 11766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:41.981040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:39585)
0620 09:21:41.981414 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:41.981447 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":51,"threads_started":1}
I20260620 09:21:42.981945 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:42.981700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:33827)
0620 09:21:42.981838 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:42.981883 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260620 09:21:42.982234 11770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:42.981917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:56293)
0620 09:21:42.982124 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:42.982184 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":46,"threads_started":1}
I20260620 09:21:42.982249 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:42.981807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:49039)
0620 09:21:42.982039 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:42.982204 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":60,"threads_started":1}
I20260620 09:21:43.982925 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:43.982732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:38211)
0620 09:21:43.982839 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:43.982875 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":61,"threads_started":1}
I20260620 09:21:43.982925 11772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:43.982674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:58243)
0620 09:21:43.982836 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:43.982874 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":53,"threads_started":1}
I20260620 09:21:43.982959 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:43.982491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:51283)
0620 09:21:43.982871 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:43.982911 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":61,"threads_started":1}
I20260620 09:21:43.986573 11774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0990b69bade74124ab980ca951f7a9ad)
I20260620 09:21:43.986649 11774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } }
I20260620 09:21:43.986792 11774 leader_election.cc:288] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0990b69bade74124ab980ca951f7a9ad (127.4.59.62:46555)
I20260620 09:21:43.987037 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:43.986880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:53734)
0620 09:21:43.986976 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:43.987009 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":43,"threads_started":1}
W20260620 09:21:43.987140 11019 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260620 09:21:43.987520 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:43.987437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:53750)
0620 09:21:43.987480 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:43.987505 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 09:21:43.987596 11019 leader_election.cc:334] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0990b69bade74124ab980ca951f7a9ad (127.4.59.62:46555): Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
I20260620 09:21:43.987651 11019 leader_election.cc:302] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 849c62a684684348ba2a21bfa6a20637; no voters: 0990b69bade74124ab980ca951f7a9ad
I20260620 09:21:43.987733 11774 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 09:21:44.983650 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:44.983480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:43739)
0620 09:21:44.983575 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:44.983598 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":43,"threads_started":1}
I20260620 09:21:44.983705 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:44.983442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:57671)
0620 09:21:44.983553 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:44.983598 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":56,"threads_started":1}
I20260620 09:21:44.983723 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:44.983386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:46369)
0620 09:21:44.983632 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:44.983672 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":89,"threads_started":1}
I20260620 09:21:45.984323 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:45.984094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:37657)
0620 09:21:45.984229 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:45.984265 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260620 09:21:45.984323 11781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:45.984099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:57461)
0620 09:21:45.984230 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:45.984269 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
I20260620 09:21:45.984387 11782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:45.984204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:46879)
0620 09:21:45.984319 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:45.984347 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":51,"threads_started":1}
I20260620 09:21:46.437980 11783 raft_consensus.cc:482] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0990b69bade74124ab980ca951f7a9ad)
I20260620 09:21:46.438104 11783 raft_consensus.cc:504] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } }
I20260620 09:21:46.438308 11783 leader_election.cc:288] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0990b69bade74124ab980ca951f7a9ad (127.4.59.62:46555)
I20260620 09:21:46.438723 11784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:46.438519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:53916)
0620 09:21:46.438637 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:46.438686 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":68,"threads_started":1}
I20260620 09:21:46.439122 11784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:46.439022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:53918)
0620 09:21:46.439072 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:46.439098 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 09:21:46.439203 11019 leader_election.cc:334] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0990b69bade74124ab980ca951f7a9ad (127.4.59.62:46555): Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
I20260620 09:21:46.439236 11019 leader_election.cc:302] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 849c62a684684348ba2a21bfa6a20637; no voters: 0990b69bade74124ab980ca951f7a9ad
I20260620 09:21:46.439354 11783 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 09:21:46.985018 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:46.984801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:56819)
0620 09:21:46.984914 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:46.984958 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":58,"threads_started":1}
I20260620 09:21:46.985061 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:46.984823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:35853)
0620 09:21:46.984992 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:46.985019 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":64,"threads_started":1}
I20260620 09:21:46.985127 11788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:46.984934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:52103)
0620 09:21:46.985041 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:46.985075 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":67,"threads_started":1}
I20260620 09:21:47.985883 11791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:47.985683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:39153)
0620 09:21:47.985785 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:47.985827 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
I20260620 09:21:47.985971 11790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:47.985671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:36775)
0620 09:21:47.985892 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:47.985925 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":44,"threads_started":1}
I20260620 09:21:47.985971 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:47.985492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:48433)
0620 09:21:47.985723 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:47.985918 (+ 195us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":70,"threads_started":1}
I20260620 09:21:48.986559 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:48.986341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:36217)
0620 09:21:48.986444 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:48.986488 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":62,"threads_started":1}
I20260620 09:21:48.986610 11793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:48.986366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:47149)
0620 09:21:48.986504 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:48.986541 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
I20260620 09:21:48.986651 11794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:48.986404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:37027)
0620 09:21:48.986579 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:48.986609 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":39,"threads_started":1}
I20260620 09:21:49.987287 11797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:49.987030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:50835)
0620 09:21:49.987205 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:49.987232 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":113,"threads_started":1}
I20260620 09:21:49.987299 11795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:49.987038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:43741)
0620 09:21:49.987179 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:49.987215 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260620 09:21:49.987339 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:49.987035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:59487)
0620 09:21:49.987171 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:49.987295 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260620 09:21:50.552425 11798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0990b69bade74124ab980ca951f7a9ad)
I20260620 09:21:50.552524 11798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } }
I20260620 09:21:50.552690 11798 leader_election.cc:288] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0990b69bade74124ab980ca951f7a9ad (127.4.59.62:46555)
I20260620 09:21:50.553020 11799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:50.552843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:38234)
0620 09:21:50.552945 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:50.552984 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":46,"threads_started":1}
W20260620 09:21:50.553108 11019 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260620 09:21:50.553568 11799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:50.553450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:38246)
0620 09:21:50.553512 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:50.553544 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260620 09:21:50.553681 11019 leader_election.cc:334] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0990b69bade74124ab980ca951f7a9ad (127.4.59.62:46555): Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
I20260620 09:21:50.553782 11019 leader_election.cc:302] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 849c62a684684348ba2a21bfa6a20637; no voters: 0990b69bade74124ab980ca951f7a9ad
I20260620 09:21:50.553884 11798 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 09:21:50.987915 11802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:50.987676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:48413)
0620 09:21:50.987828 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:50.987863 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260620 09:21:50.987915 11803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:50.987676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:41451)
0620 09:21:50.987828 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:50.987862 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":64,"threads_started":1}
I20260620 09:21:50.987975 11801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:50.987676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:48889)
0620 09:21:50.987811 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:50.987921 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
I20260620 09:21:51.988564 11804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:51.988308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:35763)
0620 09:21:51.988456 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:51.988505 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260620 09:21:51.988737 11805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:51.988556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:51177)
0620 09:21:51.988649 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:51.988687 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":54,"threads_started":1}
I20260620 09:21:51.988977 11806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:51.988723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:57155)
0620 09:21:51.988905 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:51.988944 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":44,"threads_started":1}
I20260620 09:21:52.989416 11807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:52.989190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:39335)
0620 09:21:52.989314 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:52.989355 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260620 09:21:52.989449 11808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:52.989241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:60213)
0620 09:21:52.989360 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:52.989395 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":51,"threads_started":1}
I20260620 09:21:52.989671 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:52.989515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:50395)
0620 09:21:52.989610 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:52.989638 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":45,"threads_started":1}
I20260620 09:21:53.990258 11810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:53.989832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:45597)
0620 09:21:53.990163 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:53.990203 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":88,"threads_started":1}
I20260620 09:21:53.990252 11812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:53.990017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:46035)
0620 09:21:53.990145 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:53.990188 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260620 09:21:53.990445 11811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:53.990017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:48391)
0620 09:21:53.990371 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:53.990404 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":58,"threads_started":1}
I20260620 09:21:54.991061 11814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:54.990764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:54337)
0620 09:21:54.990997 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:54.991026 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":79,"threads_started":1}
I20260620 09:21:54.991056 11813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:54.990743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:36711)
0620 09:21:54.990953 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:54.990998 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":93,"threads_started":1}
I20260620 09:21:54.991182 11815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:54.990946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:60827)
0620 09:21:54.991079 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:54.991113 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":54,"threads_started":1}
I20260620 09:21:55.991868 11817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:55.991647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:43263)
0620 09:21:55.991766 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:55.991810 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260620 09:21:55.991885 11816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:55.991604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:38363)
0620 09:21:55.991821 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:55.991848 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":67,"threads_started":1}
I20260620 09:21:55.991876 11818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:55.991676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:59543)
0620 09:21:55.991790 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:55.991826 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":41,"threads_started":1}
I20260620 09:21:56.992565 11819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:56.992341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:57451)
0620 09:21:56.992461 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:56.992501 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260620 09:21:56.992673 11820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:56.992377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:57413)
0620 09:21:56.992567 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:56.992606 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":57,"threads_started":1}
I20260620 09:21:56.992712 11821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:56.992377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:49883)
0620 09:21:56.992579 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:56.992609 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":55,"threads_started":1}
I20260620 09:21:57.993348 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:57.993092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:44291)
0620 09:21:57.993250 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:57.993290 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260620 09:21:57.993368 11824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:57.993123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:49411)
0620 09:21:57.993279 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:57.993317 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":79,"threads_started":1}
I20260620 09:21:57.993460 11822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:57.993060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:58831)
0620 09:21:57.993385 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:57.993419 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":60,"threads_started":1}
I20260620 09:21:58.994127 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:58.993910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:54413)
0620 09:21:58.994022 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:58.994065 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":40,"threads_started":1}
I20260620 09:21:58.994134 11825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:58.993831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:42009)
0620 09:21:58.994041 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:58.994081 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":65,"threads_started":1}
I20260620 09:21:58.994166 11827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:58.993997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:46647)
0620 09:21:58.994100 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:58.994125 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":54,"threads_started":1}
I20260620 09:21:59.994791 11828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:59.994549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:38673)
0620 09:21:59.994690 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:59.994730 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":61,"threads_started":1}
I20260620 09:21:59.995025 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:59.994547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:46101)
0620 09:21:59.994936 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:59.994979 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":65,"threads_started":1}
I20260620 09:21:59.995152 11830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:21:59.994876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:52913)
0620 09:21:59.995055 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 09:21:59.995105 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":66,"threads_started":1}
I20260620 09:22:00.995568 11831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:00.995294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:38491)
0620 09:22:00.995460 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:00.995509 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":69,"threads_started":1}
I20260620 09:22:00.996012 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:00.995709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:52391)
0620 09:22:00.995873 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:00.995965 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":59,"threads_started":1}
I20260620 09:22:00.996042 11832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:00.995559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:53487)
0620 09:22:00.995952 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:00.995990 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":55,"threads_started":1}
I20260620 09:22:01.996392 11834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:01.996089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:53143)
0620 09:22:01.996287 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:01.996332 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":62,"threads_started":1}
I20260620 09:22:01.996701 11835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:01.996394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:34779)
0620 09:22:01.996617 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:01.996653 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":64,"threads_started":1}
I20260620 09:22:01.996948 11836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:01.996675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:54093)
0620 09:22:01.996797 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:01.996900 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":42,"threads_started":1}
I20260620 09:22:02.272981 11837 raft_consensus.cc:482] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0990b69bade74124ab980ca951f7a9ad)
I20260620 09:22:02.273080 11837 raft_consensus.cc:504] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0990b69bade74124ab980ca951f7a9ad" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46555 } } peers { permanent_uuid: "849c62a684684348ba2a21bfa6a20637" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 38809 } attrs { promote: false } } peers { permanent_uuid: "b6318a47e1b447d3b85bf090d0a3f762" member_type: NON_VOTER last_known_addr { host: "127.4.59.60" port: 44935 } attrs { promote: true } }
I20260620 09:22:02.273301 11837 leader_election.cc:288] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0990b69bade74124ab980ca951f7a9ad (127.4.59.62:46555)
I20260620 09:22:02.273738 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:02.273519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:52524)
0620 09:22:02.273645 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:02.273690 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":83,"threads_started":1}
W20260620 09:22:02.273829 11019 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
I20260620 09:22:02.274219 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:02.274091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.0.0.1:52534)
0620 09:22:02.274166 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:02.274197 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260620 09:22:02.274297 11019 leader_election.cc:334] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0990b69bade74124ab980ca951f7a9ad (127.4.59.62:46555): Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
I20260620 09:22:02.274379 11019 leader_election.cc:302] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 849c62a684684348ba2a21bfa6a20637; no voters: 0990b69bade74124ab980ca951f7a9ad
I20260620 09:22:02.274490 11837 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 849c62a684684348ba2a21bfa6a20637 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 09:22:02.997360 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:02.996945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.1:48979)
0620 09:22:02.997260 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:02.997303 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":59,"threads_started":1}
I20260620 09:22:02.997651 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:02.997310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.3:38891)
0620 09:22:02.997407 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:02.997492 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":44,"threads_started":1}
I20260620 09:22:02.997826 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:02.997502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:46555 (local address 127.4.59.2:43489)
0620 09:22:02.997737 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:02.997771 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":43,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:903: Failure
Failed
Bad status: Timed out: Could not connect to the cluster: ConnectToClusterRpc(addrs: 127.4.59.62:46555,127.4.59.61:38809,127.4.59.60:44935, num_attempts: 243) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.4.59.62:46555,127.4.59.61:38809,127.4.59.60:44935, num_attempts: 1)
I20260620 09:22:03.165913 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 11182
I20260620 09:22:03.170683 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 11410
I20260620 09:22:03.175251 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 11546
I20260620 09:22:03.180599 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 10902
I20260620 09:22:03.185427 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 11089
2026-06-20T09:22:03Z chronyd exiting
I20260620 09:22:03.201566 4332 test_util.cc:175] -----------------------------------------------
I20260620 09:22:03.201620 4332 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781947262065055-4332-0
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (33014 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:22:03 dist-test-slave-414w krb5kdc[11845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:22:03 dist-test-slave-414w krb5kdc[11845](info): set up 2 sockets
Jun 20 09:22:03 dist-test-slave-414w krb5kdc[11845](info): commencing operation
krb5kdc: starting...
W20260620 09:22:04.245756 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:22:04 dist-test-slave-414w krb5kdc[11845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:22:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:22:04Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.62.keytab.
Entry for principal oryx/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.62.keytab.
I20260620 09:22:04.361043 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:35907
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.62.keytab
--principal=oryx/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:35907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:22:04.444083 11858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:04.444334 11858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:04.444371 11858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:04.445564 11858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:22:04.445636 11858 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 09:22:04.445667 11858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:04.445698 11858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:04.445729 11858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:22:04.445753 11858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:22:04.447232 11858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:35907
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.62.keytab
--principal=oryx/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:35907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:04.447505 11858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:04.447737 11858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:04.452760 11866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:04.452804 11863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:04.452783 11864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:04.452890 11858 server_base.cc:1034] running on GCE node
I20260620 09:22:04.453392 11858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:04.453678 11858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:04.454849 11858 hybrid_clock.cc:648] HybridClock initialized: now 1781947324454835 us; error 45 us; skew 500 ppm
Jun 20 09:22:04 dist-test-slave-414w krb5kdc[11845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947324, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:22:04.457340 11858 init.cc:377] Logged in from keytab as oryx/127.4.59.62@KRBTEST.COM (short username oryx)
I20260620 09:22:04.457810 11858 webserver.cc:460] Webserver started at http://127.4.59.62:42031/ using document root <none> and password file <none>
I20260620 09:22:04.458057 11858 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:04.458123 11858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:04.458274 11858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:04.459318 11858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "d9e18068afe5425894cf2bd7dedb13dd"
format_stamp: "Formatted at 2026-06-20 09:22:04 on dist-test-slave-414w"
server_key: "92f79a18bc257ea1e732356077160b8e"
server_key_iv: "f1b8c0ee3d94b8d886df7652e35ab0cf"
server_key_version: "encryptionkey@0"
I20260620 09:22:04.459676 11858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "d9e18068afe5425894cf2bd7dedb13dd"
format_stamp: "Formatted at 2026-06-20 09:22:04 on dist-test-slave-414w"
server_key: "92f79a18bc257ea1e732356077160b8e"
server_key_iv: "f1b8c0ee3d94b8d886df7652e35ab0cf"
server_key_version: "encryptionkey@0"
I20260620 09:22:04.461031 11858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:22:04.461854 11873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:04.462037 11858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:22:04.462128 11858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "d9e18068afe5425894cf2bd7dedb13dd"
format_stamp: "Formatted at 2026-06-20 09:22:04 on dist-test-slave-414w"
server_key: "92f79a18bc257ea1e732356077160b8e"
server_key_iv: "f1b8c0ee3d94b8d886df7652e35ab0cf"
server_key_version: "encryptionkey@0"
I20260620 09:22:04.462209 11858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:04.484634 11858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:04.486743 11858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:04.486856 11858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:04.491539 11858 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:35907
I20260620 09:22:04.491586 11935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:35907 every 8 connection(s)
I20260620 09:22:04.491925 11858 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:22:04.492544 11936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:04.494894 11936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd: Bootstrap starting.
I20260620 09:22:04.495502 11936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:04.495802 11936 log.cc:824] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd: Log is configured to *not* fsync() on all Append() calls
I20260620 09:22:04.496551 11936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd: No bootstrap required, opened a new log
I20260620 09:22:04.496541 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 11858
I20260620 09:22:04.496651 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:22:04.496865 4332 external_mini_cluster.cc:1428] Setting key b8ddb032960f548bcd181f4a5d3c21a4
I20260620 09:22:04.499004 11936 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e18068afe5425894cf2bd7dedb13dd" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35907 } }
I20260620 09:22:04.499169 11936 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:04.499253 11936 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9e18068afe5425894cf2bd7dedb13dd, State: Initialized, Role: FOLLOWER
I20260620 09:22:04.499410 11936 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e18068afe5425894cf2bd7dedb13dd" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35907 } }
I20260620 09:22:04.499495 11936 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:22:04.499521 11936 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:22:04.499583 11936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:04.500094 11936 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e18068afe5425894cf2bd7dedb13dd" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35907 } }
I20260620 09:22:04.500217 11936 leader_election.cc:302] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9e18068afe5425894cf2bd7dedb13dd; no voters:
I20260620 09:22:04.500382 11936 leader_election.cc:288] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:22:04.500422 11941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:22:04.500590 11941 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [term 1 LEADER]: Becoming Leader. State: Replica: d9e18068afe5425894cf2bd7dedb13dd, State: Running, Role: LEADER
I20260620 09:22:04.500623 11936 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:22:04.500730 11941 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e18068afe5425894cf2bd7dedb13dd" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35907 } }
I20260620 09:22:04.501044 11941 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9e18068afe5425894cf2bd7dedb13dd. Latest consensus state: current_term: 1 leader_uuid: "d9e18068afe5425894cf2bd7dedb13dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e18068afe5425894cf2bd7dedb13dd" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35907 } } }
I20260620 09:22:04.501053 11942 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9e18068afe5425894cf2bd7dedb13dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e18068afe5425894cf2bd7dedb13dd" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35907 } } }
I20260620 09:22:04.501219 11942 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [sys.catalog]: This master's current role is: LEADER
W20260620 09:22:04.502679 11957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:22:04.502741 11957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:22:04.503192 11941 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd [sys.catalog]: This master's current role is: LEADER
I20260620 09:22:04.503331 11948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:22:04.503513 11948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:22:04.505059 11948 catalog_manager.cc:1347] Generated new cluster ID: 2bd294f20829474facacdabec7db2675
I20260620 09:22:04.505108 11948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 09:22:04 dist-test-slave-414w krb5kdc[11845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM
I20260620 09:22:04.510331 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:04.497424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39848 (local address 127.4.59.62:35907)
0620 09:22:04.497605 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:04.497613 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:04.497653 (+ 40us) server_negotiation.cc:407] Connection header received
0620 09:22:04.498986 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:04.498992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:04.499037 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:04.499193 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:04.499931 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.501322 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.502357 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.502912 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.503028 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:04.506100 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:04.506104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:04.506105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:04.506119 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:04.508681 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:04.509066 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:04.509069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:04.509070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:04.509218 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:04.509427 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:04.509428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:04.509428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:04.509750 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:04.509919 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:04.509990 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 09:22:04.510136 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260620 09:22:04.521332 11948 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:22:04.522042 11948 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:22:04.526062 11948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd: Generated new TSK 0
I20260620 09:22:04.526237 11948 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.1.keytab.
Entry for principal oryx/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.1.keytab.
I20260620 09:22:04.552690 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35907
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.1.keytab
--principal=oryx/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:22:04.633666 11962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:04.633886 11962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:04.633924 11962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:04.635063 11962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:04.635128 11962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:04.635161 11962 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 09:22:04.635201 11962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:22:04.636652 11962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.1.keytab
--principal=oryx/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.59.62:35907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:04.636904 11962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:04.637130 11962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:04.642009 11970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:04.642020 11968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:04.642020 11967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:04.642272 11962 server_base.cc:1034] running on GCE node
I20260620 09:22:04.642482 11962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:04.642750 11962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:04.643895 11962 hybrid_clock.cc:648] HybridClock initialized: now 1781947324643890 us; error 35 us; skew 500 ppm
Jun 20 09:22:04 dist-test-slave-414w krb5kdc[11845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947324, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:22:04.646281 11962 init.cc:377] Logged in from keytab as oryx/127.4.59.1@KRBTEST.COM (short username oryx)
I20260620 09:22:04.646730 11962 webserver.cc:460] Webserver started at http://127.4.59.1:34249/ using document root <none> and password file <none>
I20260620 09:22:04.646946 11962 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:04.647017 11962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:04.647176 11962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:04.648190 11962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "d6952e6c79a6475fb07747c4a4ace93e"
format_stamp: "Formatted at 2026-06-20 09:22:04 on dist-test-slave-414w"
server_key: "c3b877cdcfeefd866e454d1c45d9ddd4"
server_key_iv: "bbb3e05b36f3fe4d7f8f83a6fe41dec4"
server_key_version: "encryptionkey@0"
I20260620 09:22:04.648540 11962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "d6952e6c79a6475fb07747c4a4ace93e"
format_stamp: "Formatted at 2026-06-20 09:22:04 on dist-test-slave-414w"
server_key: "c3b877cdcfeefd866e454d1c45d9ddd4"
server_key_iv: "bbb3e05b36f3fe4d7f8f83a6fe41dec4"
server_key_version: "encryptionkey@0"
I20260620 09:22:04.649924 11962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:22:04.650682 11977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:04.650862 11962 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:04.650980 11962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "d6952e6c79a6475fb07747c4a4ace93e"
format_stamp: "Formatted at 2026-06-20 09:22:04 on dist-test-slave-414w"
server_key: "c3b877cdcfeefd866e454d1c45d9ddd4"
server_key_iv: "bbb3e05b36f3fe4d7f8f83a6fe41dec4"
server_key_version: "encryptionkey@0"
I20260620 09:22:04.651057 11962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:04.662048 11962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:04.663978 11962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:04.664136 11962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:04.664672 11962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:04.664729 11962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:04.664769 11962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:04.664789 11962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 09:22:04 dist-test-slave-414w krb5kdc[11845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947324, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM
I20260620 09:22:04.672247 11962 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:39897
I20260620 09:22:04.672348 12092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:39897 every 8 connection(s)
I20260620 09:22:04.672653 11962 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:22:04.674593 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:04.665074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:42325 (local address 127.4.59.62:35907)
0620 09:22:04.665129 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:04.665131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:04.665403 (+ 272us) server_negotiation.cc:407] Connection header received
0620 09:22:04.666177 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:04.666179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:04.666191 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:04.666250 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:04.667165 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.667667 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.668780 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.668986 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.669069 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:04.671591 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:04.671593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:04.671593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:04.671604 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:04.673389 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:04.673776 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:04.673777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:04.673777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:04.673830 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:04.674039 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:04.674040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:04.674040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:04.674140 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:04.674193 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:04.674464 (+ 271us) server_negotiation.cc:299] Negotiation successful
0620 09:22:04.674530 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 09:22:04.674830 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:04.665056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35907 (local address 127.4.59.1:42325)
0620 09:22:04.665240 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:04.665298 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:04.666034 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:04.666355 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:04.666362 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:04.666765 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:04.667068 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:04.667075 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.667768 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.667769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:04.668680 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:04.668682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.669080 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.669082 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:04.669260 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:04.669808 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:04.669818 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:04.671481 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:04.673481 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:04.673488 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:04.673490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:04.673704 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:04.673875 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:04.673876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:04.673876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:04.673961 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:04.674225 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:04.674232 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:04.674346 (+ 114us) client_negotiation.cc:769] Sending connection context
0620 09:22:04.674441 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 09:22:04.674561 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260620 09:22:04.675282 12093 heartbeater.cc:347] Connected to a master server at 127.4.59.62:35907
I20260620 09:22:04.675362 12093 heartbeater.cc:464] Registering TS with master...
I20260620 09:22:04.675536 12093 heartbeater.cc:511] Master 127.4.59.62:35907 requested a full tablet report, sending...
I20260620 09:22:04.676069 11889 ts_manager.cc:194] Registered new tserver with Master: d6952e6c79a6475fb07747c4a4ace93e (127.4.59.1:39897)
I20260620 09:22:04.676750 11889 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:42325
I20260620 09:22:04.678390 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 11962
I20260620 09:22:04.678498 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:22:04.678683 4332 external_mini_cluster.cc:1428] Setting key e9925de7e5c4d7ac446f67366ff3f7fe
I20260620 09:22:04.682766 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:04.676347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:47761 (local address 127.4.59.62:35907)
0620 09:22:04.676932 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:04.676933 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:04.676942 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:22:04.676965 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:04.676966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:04.676973 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:04.677021 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:04.677391 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.677958 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.678898 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.679095 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.679180 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:04.680110 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:04.680111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:04.680111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:04.680117 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:04.681718 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:04.682048 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:04.682049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:04.682050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:04.682107 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:04.682301 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:04.682302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:04.682303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:04.682403 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:04.682476 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:04.682606 (+ 130us) server_negotiation.cc:299] Negotiation successful
0620 09:22:04.682676 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567}
I20260620 09:22:04.682776 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:04.676287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35907 (local address 127.4.59.1:47761)
0620 09:22:04.676398 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:04.676408 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:04.676458 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:04.677029 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:04.677030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:04.677264 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:04.677320 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:04.677321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.678055 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.678056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:04.678828 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:04.678830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.679181 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.679182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:04.679245 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:04.679631 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:04.679636 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:04.680018 (+ 382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:04.681800 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:04.681801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:04.681801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:04.681953 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:04.682174 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:04.682175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:04.682175 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:04.682231 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:04.682468 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:04.682468 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:04.682511 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 09:22:04.682579 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 09:22:04.682674 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for oryx/127.4.59.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.2.keytab.
Entry for principal oryx/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.2.keytab.
I20260620 09:22:04.704017 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35907
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.2.keytab
--principal=oryx/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:22:04.790382 12102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:04.790576 12102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:04.790599 12102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:04.791738 12102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:04.791774 12102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:04.791795 12102 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 09:22:04.791812 12102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:22:04.793339 12102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.2.keytab
--principal=oryx/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.59.62:35907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:04.793519 12102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:04.793689 12102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:04.798501 12110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:04.798534 12107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:04.798590 12102 server_base.cc:1034] running on GCE node
W20260620 09:22:04.798648 12108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:04.798864 12102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:04.799053 12102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:04.800190 12102 hybrid_clock.cc:648] HybridClock initialized: now 1781947324800175 us; error 41 us; skew 500 ppm
Jun 20 09:22:04 dist-test-slave-414w krb5kdc[11845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947324, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:22:04.802785 12102 init.cc:377] Logged in from keytab as oryx/127.4.59.2@KRBTEST.COM (short username oryx)
I20260620 09:22:04.803136 12102 webserver.cc:460] Webserver started at http://127.4.59.2:38049/ using document root <none> and password file <none>
I20260620 09:22:04.803339 12102 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:04.803380 12102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:04.803465 12102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:04.804419 12102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "39c8f9995e714f729ff000288312d4f0"
format_stamp: "Formatted at 2026-06-20 09:22:04 on dist-test-slave-414w"
server_key: "5b70e67b5fa372867887a8c0b21d4fe9"
server_key_iv: "65a1bf7d35f2c7d00ea52ed4e7f668fb"
server_key_version: "encryptionkey@0"
I20260620 09:22:04.804700 12102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "39c8f9995e714f729ff000288312d4f0"
format_stamp: "Formatted at 2026-06-20 09:22:04 on dist-test-slave-414w"
server_key: "5b70e67b5fa372867887a8c0b21d4fe9"
server_key_iv: "65a1bf7d35f2c7d00ea52ed4e7f668fb"
server_key_version: "encryptionkey@0"
I20260620 09:22:04.806005 12102 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:22:04.806699 12117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:04.806870 12102 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:22:04.806946 12102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "39c8f9995e714f729ff000288312d4f0"
format_stamp: "Formatted at 2026-06-20 09:22:04 on dist-test-slave-414w"
server_key: "5b70e67b5fa372867887a8c0b21d4fe9"
server_key_iv: "65a1bf7d35f2c7d00ea52ed4e7f668fb"
server_key_version: "encryptionkey@0"
I20260620 09:22:04.806998 12102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:04.827401 12102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:04.829329 12102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:04.829435 12102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:04.829864 12102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:04.829897 12102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:04.829948 12102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:04.829980 12102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:04.836915 12102 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:40301
I20260620 09:22:04.836947 12232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:40301 every 8 connection(s)
I20260620 09:22:04.837357 12102 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
Jun 20 09:22:04 dist-test-slave-414w krb5kdc[11845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947324, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM
I20260620 09:22:04.839246 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 12102
I20260620 09:22:04.839361 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:22:04.839566 4332 external_mini_cluster.cc:1428] Setting key 715acc51758958ac52ad82ea983765c3
I20260620 09:22:04.841324 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:04.830461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:49689 (local address 127.4.59.62:35907)
0620 09:22:04.830524 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:04.830526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:04.830749 (+ 223us) server_negotiation.cc:407] Connection header received
0620 09:22:04.831293 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:04.831295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:04.831306 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:04.831382 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:04.832339 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.833170 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.834835 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.835135 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.835193 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:04.837948 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:04.837949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:04.837949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:04.837956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:04.839566 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:04.840043 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:04.840044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:04.840044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:04.840108 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:04.840618 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:04.840619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:04.840619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:04.840747 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:04.840823 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:04.841098 (+ 275us) server_negotiation.cc:299] Negotiation successful
0620 09:22:04.841236 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 09:22:04.841483 12126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:04.830401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35907 (local address 127.4.59.2:49689)
0620 09:22:04.830604 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:04.830642 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:04.831167 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:04.831506 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:04.831512 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:04.831926 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:04.832224 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:04.832232 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.833279 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.833280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:04.834739 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:04.834742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.835221 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.835223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:04.835393 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:04.835978 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:04.835988 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:04.837859 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:04.839665 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:04.839672 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:04.839674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:04.839972 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:04.840277 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:04.840279 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:04.840280 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:04.840525 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:04.840830 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:04.840836 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:04.840966 (+ 130us) client_negotiation.cc:769] Sending connection context
0620 09:22:04.841064 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 09:22:04.841258 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":76,"threads_started":1}
I20260620 09:22:04.841964 12233 heartbeater.cc:347] Connected to a master server at 127.4.59.62:35907
I20260620 09:22:04.842063 12233 heartbeater.cc:464] Registering TS with master...
I20260620 09:22:04.842245 12233 heartbeater.cc:511] Master 127.4.59.62:35907 requested a full tablet report, sending...
I20260620 09:22:04.842617 11890 ts_manager.cc:194] Registered new tserver with Master: 39c8f9995e714f729ff000288312d4f0 (127.4.59.2:40301)
I20260620 09:22:04.843111 11890 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:49689
WARNING: no policy specified for oryx/127.4.59.3@KRBTEST.COM; defaulting to no policy
I20260620 09:22:04.848606 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:04.843062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:40937 (local address 127.4.59.62:35907)
0620 09:22:04.843097 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:04.843099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:04.843215 (+ 116us) server_negotiation.cc:407] Connection header received
0620 09:22:04.843239 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:04.843240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:04.843250 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:04.843303 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:04.843662 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.844120 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.844957 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.845222 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.845311 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:04.846203 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:04.846204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:04.846205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:04.846211 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:04.847496 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:04.847829 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:04.847831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:04.847831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:04.847882 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:04.848082 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:04.848083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:04.848083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:04.848222 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:04.848300 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:04.848444 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 09:22:04.848515 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 09:22:04.848620 12240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:04.843000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35907 (local address 127.4.59.2:40937)
0620 09:22:04.843126 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:04.843137 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:04.843193 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:04.843322 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:04.843323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:04.843511 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:04.843573 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:04.843574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.844317 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.844318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:04.844862 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:04.844863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:04.845312 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:04.845313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:04.845381 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:04.845822 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:04.845827 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:04.846097 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:04.847577 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:04.847578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:04.847579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:04.847746 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:04.847954 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:04.847955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:04.847956 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:04.848011 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:04.848292 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:04.848292 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:04.848334 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 09:22:04.848420 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 09:22:04.848515 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":5,"thread_start_us":66,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.3.keytab.
Entry for principal oryx/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.3.keytab.
I20260620 09:22:04.865540 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35907
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.3.keytab
--principal=oryx/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:22:04.945057 12242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:04.945317 12242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:04.945343 12242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:04.946453 12242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:04.946486 12242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:04.946506 12242 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 09:22:04.946523 12242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:22:04.947926 12242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.3.keytab
--principal=oryx/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.59.62:35907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:04.948112 12242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:04.948282 12242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:04.953243 12248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:04.953485 12242 server_base.cc:1034] running on GCE node
W20260620 09:22:04.953403 12250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:04.953416 12247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:04.953902 12242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:04.954167 12242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:04.955334 12242 hybrid_clock.cc:648] HybridClock initialized: now 1781947324955324 us; error 44 us; skew 500 ppm
Jun 20 09:22:04 dist-test-slave-414w krb5kdc[11845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947324, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:22:04.958098 12242 init.cc:377] Logged in from keytab as oryx/127.4.59.3@KRBTEST.COM (short username oryx)
I20260620 09:22:04.958545 12242 webserver.cc:460] Webserver started at http://127.4.59.3:41857/ using document root <none> and password file <none>
I20260620 09:22:04.958798 12242 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:04.958868 12242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:04.959018 12242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:04.960074 12242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "c09e2e49b59d443c8d7b25cce0229255"
format_stamp: "Formatted at 2026-06-20 09:22:04 on dist-test-slave-414w"
server_key: "612fda1ebcd729abf65df062b4f9661f"
server_key_iv: "a847ace661dc76bfef3970cb516ef29e"
server_key_version: "encryptionkey@0"
I20260620 09:22:04.960444 12242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "c09e2e49b59d443c8d7b25cce0229255"
format_stamp: "Formatted at 2026-06-20 09:22:04 on dist-test-slave-414w"
server_key: "612fda1ebcd729abf65df062b4f9661f"
server_key_iv: "a847ace661dc76bfef3970cb516ef29e"
server_key_version: "encryptionkey@0"
I20260620 09:22:04.961905 12242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260620 09:22:04.962908 12257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:04.963224 12242 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:22:04.963330 12242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "c09e2e49b59d443c8d7b25cce0229255"
format_stamp: "Formatted at 2026-06-20 09:22:04 on dist-test-slave-414w"
server_key: "612fda1ebcd729abf65df062b4f9661f"
server_key_iv: "a847ace661dc76bfef3970cb516ef29e"
server_key_version: "encryptionkey@0"
I20260620 09:22:04.963405 12242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:04.995115 12242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:04.997416 12242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:04.997570 12242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:04.998067 12242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:04.998133 12242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:04.998302 12242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:04.998360 12242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:05.005503 12242 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:35377
I20260620 09:22:05.005628 12372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:35377 every 8 connection(s)
I20260620 09:22:05.005935 12242 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
Jun 20 09:22:05 dist-test-slave-414w krb5kdc[11845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947324, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM
I20260620 09:22:05.008819 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:04.998662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:59727 (local address 127.4.59.62:35907)
0620 09:22:04.998716 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:04.998718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:04.998922 (+ 204us) server_negotiation.cc:407] Connection header received
0620 09:22:04.999416 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:04.999417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:04.999432 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:04.999512 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:05.000335 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.001298 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.002913 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.003267 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.003350 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.006513 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:05.006514 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:05.006515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:05.006522 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:05.007660 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:05.007999 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:05.008000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:05.008001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:05.008070 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:05.008300 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:05.008301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:05.008301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:05.008430 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:05.008494 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.008695 (+ 201us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.008759 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 09:22:05.009012 12266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:04.998673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35907 (local address 127.4.59.3:59727)
0620 09:22:04.998809 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:04.998843 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:04.999312 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:04.999561 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:04.999564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:04.999901 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:05.000218 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.000225 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.001410 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.001411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.002808 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.002810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.003367 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.003368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.003535 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.004649 (+ 1114us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:05.004659 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:05.006414 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:05.007726 (+ 1312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:05.007731 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:05.007733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:05.007934 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:05.008129 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:05.008129 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:05.008130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:05.008220 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:05.008494 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:05.008498 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:05.008602 (+ 104us) client_negotiation.cc:769] Sending connection context
0620 09:22:05.008680 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 09:22:05.008801 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260620 09:22:05.009477 12373 heartbeater.cc:347] Connected to a master server at 127.4.59.62:35907
I20260620 09:22:05.009585 12373 heartbeater.cc:464] Registering TS with master...
I20260620 09:22:05.009769 12373 heartbeater.cc:511] Master 127.4.59.62:35907 requested a full tablet report, sending...
I20260620 09:22:05.010140 11889 ts_manager.cc:194] Registered new tserver with Master: c09e2e49b59d443c8d7b25cce0229255 (127.4.59.3:35377)
I20260620 09:22:05.010428 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 12242
I20260620 09:22:05.010535 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:22:05.010674 11889 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:59727
I20260620 09:22:05.010748 4332 external_mini_cluster.cc:1428] Setting key 4b05f03496fd0381dc77da489ed34c35
I20260620 09:22:05.011462 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:22:05.016526 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.010546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:41535 (local address 127.4.59.62:35907)
0620 09:22:05.010797 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.010799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.010969 (+ 170us) server_negotiation.cc:407] Connection header received
0620 09:22:05.010991 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.010992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.011002 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.011049 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:05.011808 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.012264 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.012957 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.013209 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.013291 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.014161 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:05.014163 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:05.014163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:05.014169 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:05.015467 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:05.015845 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:05.015846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:05.015846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:05.015902 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:05.016114 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:05.016114 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:05.016115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:05.016206 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:05.016256 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.016415 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.016471 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231}
I20260620 09:22:05.016670 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.010665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35907 (local address 127.4.59.3:41535)
0620 09:22:05.010859 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:05.010870 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:05.010933 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:05.011055 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:05.011056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:05.011675 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:05.011720 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.011721 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.012356 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.012357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.012862 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.012863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.013292 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.013293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.013358 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.013792 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:05.013797 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:05.014056 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:05.015559 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:05.015560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:05.015561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:05.015763 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:05.015975 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:05.015976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:05.015977 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:05.016048 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:05.016271 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:05.016272 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:05.016319 (+ 47us) client_negotiation.cc:769] Sending connection context
0620 09:22:05.016392 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 09:22:05.016504 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":106,"threads_started":1}
Jun 20 09:22:05 dist-test-slave-414w krb5kdc[11845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947325, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 09:22:05 dist-test-slave-414w krb5kdc[11845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:05 dist-test-slave-414w krb5kdc[11845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
W20260620 09:22:05.032145 11940 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:05.027081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39856 (local address 127.4.59.62:35907)
0620 09:22:05.027141 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.027142 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.027150 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:22:05.027221 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.027222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.027230 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.027281 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:05.027787 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.028236 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.028944 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.029113 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.029231 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.032047 (+ 2816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39856: BlockingRecv error: recv got EOF from 127.0.0.1:39856 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 20 09:22:05 dist-test-slave-414w krb5kdc[11845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947325, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM
I20260620 09:22:05.039024 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.032845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39872 (local address 127.4.59.62:35907)
0620 09:22:05.032887 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.032888 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.032930 (+ 42us) server_negotiation.cc:407] Connection header received
0620 09:22:05.032988 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.032989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.032996 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.033043 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:05.033520 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.033979 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.034620 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.034805 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.034859 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.036834 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:05.036835 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:05.036835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:05.036841 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:05.038107 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:05.038404 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:05.038405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:05.038405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:05.038452 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:05.038605 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:05.038606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:05.038606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:05.038741 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:05.038838 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.038908 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.038962 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:22:05.040647 11889 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:22:05.042557 11889 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:22:05.049103 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.044972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:35377 (local address 127.0.0.1:36110)
0620 09:22:05.045193 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:05.045320 (+ 127us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:05.045442 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:05.045687 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:05.045689 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:05.045691 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:22:05.045784 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.045786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.047184 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.047185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.048127 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.048129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.048782 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.048783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.048868 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.048882 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 09:22:05.048954 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 09:22:05.049010 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":30,"thread_start_us":60,"threads_started":1}
I20260620 09:22:05.049114 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.045064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36110 (local address 127.4.59.3:35377)
0620 09:22:05.045318 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.045324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.045403 (+ 79us) server_negotiation.cc:407] Connection header received
0620 09:22:05.045562 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.045564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.045579 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.045619 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:22:05.045877 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.047062 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.048207 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.048690 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.048780 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.048861 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.048979 (+ 118us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.049015 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":146,"threads_started":1}
I20260620 09:22:05.049443 12397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.045097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39216 (local address 127.4.59.2:40301)
0620 09:22:05.045616 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.045619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.045628 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:22:05.045663 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.045666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.045681 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.045749 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:22:05.045925 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.046684 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.048065 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.048779 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.048843 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.048913 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.049299 (+ 386us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.049347 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":86,"threads_started":1}
I20260620 09:22:05.049434 12399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.045216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:39897 (local address 127.0.0.1:43212)
0620 09:22:05.045369 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:05.045379 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:05.045442 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:05.045754 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:05.045755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:05.045756 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:22:05.045785 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.045786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.047195 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.047196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.048309 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.048311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.049234 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.049234 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.049292 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.049296 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:22:05.049327 (+ 31us) client_negotiation.cc:240] Negotiation successful
0620 09:22:05.049358 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":38,"thread_start_us":68,"threads_started":1}
I20260620 09:22:05.049559 12396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.044978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:40301 (local address 127.0.0.1:39216)
0620 09:22:05.045372 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:05.045381 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:05.045442 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:05.045817 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:05.045818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:05.045820 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:22:05.045867 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.045869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.046778 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.046779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.047968 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.047971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.048948 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.048949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.049011 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.049015 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:22:05.049064 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 09:22:05.049107 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":92,"threads_started":1}
I20260620 09:22:05.049820 12400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.045424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43212 (local address 127.4.59.1:39897)
0620 09:22:05.045573 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.045577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.045586 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:22:05.045627 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.045629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.045645 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.045744 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:22:05.046055 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.047113 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.048395 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.049111 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.049508 (+ 397us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.049628 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.049656 (+ 28us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.049712 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260620 09:22:05.050343 12027 tablet_service.cc:1467] Processing CreateTablet for tablet 7e096348d2dc44d28e89d125c949d37d (DEFAULT_TABLE table=test-table [id=fe5779a710be4137ba2ae6bce91a6cce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:22:05.050333 12167 tablet_service.cc:1467] Processing CreateTablet for tablet 7e096348d2dc44d28e89d125c949d37d (DEFAULT_TABLE table=test-table [id=fe5779a710be4137ba2ae6bce91a6cce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:22:05.050333 12305 tablet_service.cc:1467] Processing CreateTablet for tablet 7e096348d2dc44d28e89d125c949d37d (DEFAULT_TABLE table=test-table [id=fe5779a710be4137ba2ae6bce91a6cce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:22:05.050621 12027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e096348d2dc44d28e89d125c949d37d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:05.050653 12167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e096348d2dc44d28e89d125c949d37d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:05.050652 12305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e096348d2dc44d28e89d125c949d37d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:05.053851 12401 tablet_bootstrap.cc:492] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e: Bootstrap starting.
I20260620 09:22:05.054478 12403 tablet_bootstrap.cc:492] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255: Bootstrap starting.
I20260620 09:22:05.054719 12401 tablet_bootstrap.cc:654] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:05.055063 12401 log.cc:824] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e: Log is configured to *not* fsync() on all Append() calls
I20260620 09:22:05.055120 12403 tablet_bootstrap.cc:654] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:05.055438 12403 log.cc:824] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255: Log is configured to *not* fsync() on all Append() calls
I20260620 09:22:05.055739 12402 tablet_bootstrap.cc:492] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0: Bootstrap starting.
I20260620 09:22:05.055990 12401 tablet_bootstrap.cc:492] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e: No bootstrap required, opened a new log
I20260620 09:22:05.056094 12401 ts_tablet_manager.cc:1397] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:22:05.056121 12403 tablet_bootstrap.cc:492] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255: No bootstrap required, opened a new log
I20260620 09:22:05.056205 12403 ts_tablet_manager.cc:1397] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:22:05.056320 12402 tablet_bootstrap.cc:654] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:05.056625 12402 log.cc:824] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0: Log is configured to *not* fsync() on all Append() calls
I20260620 09:22:05.057349 12402 tablet_bootstrap.cc:492] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0: No bootstrap required, opened a new log
I20260620 09:22:05.057403 12402 ts_tablet_manager.cc:1397] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:22:05.058225 12403 raft_consensus.cc:348] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } }
I20260620 09:22:05.058303 12401 raft_consensus.cc:348] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } }
I20260620 09:22:05.058403 12403 raft_consensus.cc:374] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:05.058423 12401 raft_consensus.cc:374] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:05.058441 12403 raft_consensus.cc:729] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c09e2e49b59d443c8d7b25cce0229255, State: Initialized, Role: FOLLOWER
I20260620 09:22:05.058456 12401 raft_consensus.cc:729] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6952e6c79a6475fb07747c4a4ace93e, State: Initialized, Role: FOLLOWER
I20260620 09:22:05.058585 12403 consensus_queue.cc:260] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } }
I20260620 09:22:05.058606 12401 consensus_queue.cc:260] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } }
I20260620 09:22:05.058858 12401 ts_tablet_manager.cc:1428] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 09:22:05.058858 12403 ts_tablet_manager.cc:1428] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:22:05.058991 12093 heartbeater.cc:503] Master 127.4.59.62:35907 was elected leader, sending a full tablet report...
I20260620 09:22:05.059012 12402 raft_consensus.cc:348] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } }
I20260620 09:22:05.059172 12402 raft_consensus.cc:374] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:05.059206 12402 raft_consensus.cc:729] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39c8f9995e714f729ff000288312d4f0, State: Initialized, Role: FOLLOWER
I20260620 09:22:05.059283 12373 heartbeater.cc:503] Master 127.4.59.62:35907 was elected leader, sending a full tablet report...
I20260620 09:22:05.059315 12402 consensus_queue.cc:260] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } }
I20260620 09:22:05.059562 12402 ts_tablet_manager.cc:1428] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:22:05.059731 12233 heartbeater.cc:503] Master 127.4.59.62:35907 was elected leader, sending a full tablet report...
W20260620 09:22:05.088011 12234 tablet.cc:2369] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 09:22:05.173364 12094 tablet.cc:2369] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:22:05.184688 12407 raft_consensus.cc:482] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:22:05.184816 12407 raft_consensus.cc:504] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } }
I20260620 09:22:05.185133 12407 leader_election.cc:288] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6952e6c79a6475fb07747c4a4ace93e (127.4.59.1:39897), 39c8f9995e714f729ff000288312d4f0 (127.4.59.2:40301)
I20260620 09:22:05.188235 12266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.185363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:40301 (local address 127.4.59.3:56369)
0620 09:22:05.185421 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:05.185434 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:05.185480 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:05.185651 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:05.185652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:05.185654 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:22:05.185733 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.185736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.186590 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.186591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.187376 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.187378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.187938 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.187939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.188020 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.188024 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:22:05.188083 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 09:22:05.188118 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 09:22:05.188241 12397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.185466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:56369 (local address 127.4.59.2:40301)
0620 09:22:05.185507 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.185509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.185519 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:22:05.185554 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.185554 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.185566 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.185630 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:22:05.185807 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.186403 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.187467 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.187857 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.187912 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.187921 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.188097 (+ 176us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.188145 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 09:22:05.188320 12400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.185401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:47403 (local address 127.4.59.1:39897)
0620 09:22:05.185449 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.185451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.185636 (+ 185us) server_negotiation.cc:407] Connection header received
0620 09:22:05.185668 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.185669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.185682 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.185742 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:22:05.186015 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.186610 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.187553 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.187941 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.188012 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.188022 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.188179 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.188225 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 09:22:05.188355 12410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.185363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:39897 (local address 127.4.59.3:47403)
0620 09:22:05.185525 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:05.185537 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:05.185623 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:05.185752 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:05.185752 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:05.185754 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:22:05.185839 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.185841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.186700 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.186701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.187468 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.187470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.188019 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.188020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.188101 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.188105 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:22:05.188161 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 09:22:05.188198 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":12,"thread_start_us":84,"threads_started":1}
I20260620 09:22:05.188623 12187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e096348d2dc44d28e89d125c949d37d" candidate_uuid: "c09e2e49b59d443c8d7b25cce0229255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c8f9995e714f729ff000288312d4f0" is_pre_election: true
I20260620 09:22:05.188761 12187 raft_consensus.cc:2454] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c09e2e49b59d443c8d7b25cce0229255 in term 0.
I20260620 09:22:05.188963 12047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e096348d2dc44d28e89d125c949d37d" candidate_uuid: "c09e2e49b59d443c8d7b25cce0229255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6952e6c79a6475fb07747c4a4ace93e" is_pre_election: true
I20260620 09:22:05.188995 12261 leader_election.cc:302] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39c8f9995e714f729ff000288312d4f0, c09e2e49b59d443c8d7b25cce0229255; no voters:
I20260620 09:22:05.189123 12047 raft_consensus.cc:2454] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c09e2e49b59d443c8d7b25cce0229255 in term 0.
I20260620 09:22:05.189164 12407 raft_consensus.cc:2781] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:22:05.189260 12407 raft_consensus.cc:482] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:22:05.189293 12407 raft_consensus.cc:3037] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:05.190263 12407 raft_consensus.cc:504] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } }
I20260620 09:22:05.190435 12407 leader_election.cc:288] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [CANDIDATE]: Term 1 election: Requested vote from peers d6952e6c79a6475fb07747c4a4ace93e (127.4.59.1:39897), 39c8f9995e714f729ff000288312d4f0 (127.4.59.2:40301)
I20260620 09:22:05.190606 12187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e096348d2dc44d28e89d125c949d37d" candidate_uuid: "c09e2e49b59d443c8d7b25cce0229255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c8f9995e714f729ff000288312d4f0"
I20260620 09:22:05.190630 12047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e096348d2dc44d28e89d125c949d37d" candidate_uuid: "c09e2e49b59d443c8d7b25cce0229255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6952e6c79a6475fb07747c4a4ace93e"
I20260620 09:22:05.190702 12047 raft_consensus.cc:3037] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:05.190733 12187 raft_consensus.cc:3037] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:05.191368 12047 raft_consensus.cc:2454] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c09e2e49b59d443c8d7b25cce0229255 in term 1.
I20260620 09:22:05.191511 12187 raft_consensus.cc:2454] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c09e2e49b59d443c8d7b25cce0229255 in term 1.
I20260620 09:22:05.191543 12258 leader_election.cc:302] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c09e2e49b59d443c8d7b25cce0229255, d6952e6c79a6475fb07747c4a4ace93e; no voters:
I20260620 09:22:05.191617 12407 raft_consensus.cc:2781] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:22:05.191756 12407 raft_consensus.cc:686] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [term 1 LEADER]: Becoming Leader. State: Replica: c09e2e49b59d443c8d7b25cce0229255, State: Running, Role: LEADER
I20260620 09:22:05.191883 12407 consensus_queue.cc:237] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } }
I20260620 09:22:05.192756 11890 catalog_manager.cc:5495] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 reported cstate change: term changed from 0 to 1, leader changed from <none> to c09e2e49b59d443c8d7b25cce0229255 (127.4.59.3). New cstate: current_term: 1 leader_uuid: "c09e2e49b59d443c8d7b25cce0229255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } health_report { overall_health: UNKNOWN } } }
I20260620 09:22:05.234187 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.231893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36122 (local address 127.4.59.3:35377)
0620 09:22:05.231982 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.231984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.231992 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:22:05.232024 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.232025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.232042 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.232103 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:22:05.232237 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.232749 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.233530 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.233724 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.233781 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.233892 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:22:05.233980 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:22:05.234031 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.234063 (+ 32us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.234079 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 09:22:05.235858 12047 raft_consensus.cc:1261] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Refusing update from remote peer c09e2e49b59d443c8d7b25cce0229255: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:22:05.235880 12187 raft_consensus.cc:1261] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0 [term 1 FOLLOWER]: Refusing update from remote peer c09e2e49b59d443c8d7b25cce0229255: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:22:05.236228 12411 consensus_queue.cc:1035] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:22:05.236346 12407 consensus_queue.cc:1035] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:22:05.239763 12416 mvcc.cc:205] Tried to move back new op lower bound from 7298856244163530752 to 7298856243986526208. Current Snapshot: MvccSnapshot[applied={T|T < 7298856244163530752}]
I20260620 09:22:05.240721 11890 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:22:05.240814 11890 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:22:05.241598 11890 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd: Sending DeleteTablet for 3 replicas of tablet 7e096348d2dc44d28e89d125c949d37d
I20260620 09:22:05.241968 12305 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e096348d2dc44d28e89d125c949d37d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:22:05 UTC) from {username='oryx', principal='oryx/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:36110
I20260620 09:22:05.241998 12027 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e096348d2dc44d28e89d125c949d37d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:22:05 UTC) from {username='oryx', principal='oryx/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:43212
I20260620 09:22:05.242053 12167 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e096348d2dc44d28e89d125c949d37d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:22:05 UTC) from {username='oryx', principal='oryx/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:39216
I20260620 09:22:05.242177 12424 tablet_replica.cc:330] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e: stopping tablet replica
I20260620 09:22:05.242266 12424 raft_consensus.cc:2229] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:22:05.242296 12425 tablet_replica.cc:330] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255: stopping tablet replica
I20260620 09:22:05.242350 12424 raft_consensus.cc:2258] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:22:05.242424 12425 raft_consensus.cc:2229] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:22:05.242429 11890 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:22:05.242555 11890 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:22:05.242697 12425 raft_consensus.cc:2258] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:22:05.242810 12424 ts_tablet_manager.cc:1905] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:22:05.243132 12425 ts_tablet_manager.cc:1905] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:22:05.244288 12027 tablet_service.cc:1467] Processing CreateTablet for tablet f773b13495c14bc8818acad133714d3a (DEFAULT_TABLE table=test-table [id=6ab743972ed246ce9ccdaa3fc4694605]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:22:05.244395 12424 ts_tablet_manager.cc:1918] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:22:05.244478 12424 log.cc:1196] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/wal/wals/7e096348d2dc44d28e89d125c949d37d
I20260620 09:22:05.244479 12305 tablet_service.cc:1467] Processing CreateTablet for tablet f773b13495c14bc8818acad133714d3a (DEFAULT_TABLE table=test-table [id=6ab743972ed246ce9ccdaa3fc4694605]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:22:05.244567 12305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f773b13495c14bc8818acad133714d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:05.244750 12424 ts_tablet_manager.cc:1939] T 7e096348d2dc44d28e89d125c949d37d P d6952e6c79a6475fb07747c4a4ace93e: Deleting consensus metadata
I20260620 09:22:05.244769 12425 ts_tablet_manager.cc:1918] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:22:05.244827 12425 log.cc:1196] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/7e096348d2dc44d28e89d125c949d37d
I20260620 09:22:05.245070 12425 ts_tablet_manager.cc:1939] T 7e096348d2dc44d28e89d125c949d37d P c09e2e49b59d443c8d7b25cce0229255: Deleting consensus metadata
I20260620 09:22:05.244386 12027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f773b13495c14bc8818acad133714d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:05.245432 12167 tablet_service.cc:1467] Processing CreateTablet for tablet f773b13495c14bc8818acad133714d3a (DEFAULT_TABLE table=test-table [id=6ab743972ed246ce9ccdaa3fc4694605]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:22:05.245523 12167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f773b13495c14bc8818acad133714d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:05.245889 12403 tablet_bootstrap.cc:492] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255: Bootstrap starting.
I20260620 09:22:05.246279 12403 tablet_bootstrap.cc:654] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:05.246587 12401 tablet_bootstrap.cc:492] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e: Bootstrap starting.
I20260620 09:22:05.246785 12426 tablet_replica.cc:330] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0: stopping tablet replica
I20260620 09:22:05.246886 12402 tablet_bootstrap.cc:492] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0: Bootstrap starting.
I20260620 09:22:05.246953 12401 tablet_bootstrap.cc:654] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:05.246886 12426 raft_consensus.cc:2229] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:22:05.247141 12426 raft_consensus.cc:2258] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:22:05.247328 12402 tablet_bootstrap.cc:654] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:05.247376 11874 catalog_manager.cc:4841] TS c09e2e49b59d443c8d7b25cce0229255 (127.4.59.3:35377): tablet 7e096348d2dc44d28e89d125c949d37d (table test-table [id=fe5779a710be4137ba2ae6bce91a6cce]) successfully deleted
I20260620 09:22:05.247466 11874 catalog_manager.cc:4841] TS d6952e6c79a6475fb07747c4a4ace93e (127.4.59.1:39897): tablet 7e096348d2dc44d28e89d125c949d37d (table test-table [id=fe5779a710be4137ba2ae6bce91a6cce]) successfully deleted
I20260620 09:22:05.247545 12401 tablet_bootstrap.cc:492] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e: No bootstrap required, opened a new log
I20260620 09:22:05.247593 12401 ts_tablet_manager.cc:1397] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 09:22:05.247684 12426 ts_tablet_manager.cc:1905] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:22:05.247727 12401 raft_consensus.cc:348] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.247792 12401 raft_consensus.cc:374] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:05.247818 12401 raft_consensus.cc:729] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6952e6c79a6475fb07747c4a4ace93e, State: Initialized, Role: FOLLOWER
I20260620 09:22:05.247864 12401 consensus_queue.cc:260] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.247954 12401 ts_tablet_manager.cc:1428] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 09:22:05.248304 12403 tablet_bootstrap.cc:492] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255: No bootstrap required, opened a new log
I20260620 09:22:05.248363 12403 ts_tablet_manager.cc:1397] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:22:05.248497 12403 raft_consensus.cc:348] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.248561 12403 raft_consensus.cc:374] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:05.248575 12403 raft_consensus.cc:729] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c09e2e49b59d443c8d7b25cce0229255, State: Initialized, Role: FOLLOWER
I20260620 09:22:05.248639 12403 consensus_queue.cc:260] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.248728 12403 ts_tablet_manager.cc:1428] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:05.249325 12426 ts_tablet_manager.cc:1918] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:22:05.249428 12426 log.cc:1196] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/7e096348d2dc44d28e89d125c949d37d
I20260620 09:22:05.249580 12402 tablet_bootstrap.cc:492] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0: No bootstrap required, opened a new log
I20260620 09:22:05.249646 12402 ts_tablet_manager.cc:1397] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 09:22:05.249687 12426 ts_tablet_manager.cc:1939] T 7e096348d2dc44d28e89d125c949d37d P 39c8f9995e714f729ff000288312d4f0: Deleting consensus metadata
I20260620 09:22:05.249794 12402 raft_consensus.cc:348] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.249881 12402 raft_consensus.cc:374] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:05.249909 12402 raft_consensus.cc:729] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39c8f9995e714f729ff000288312d4f0, State: Initialized, Role: FOLLOWER
I20260620 09:22:05.249953 12402 consensus_queue.cc:260] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.250041 12402 ts_tablet_manager.cc:1428] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 09:22:05.250183 11877 catalog_manager.cc:4841] TS 39c8f9995e714f729ff000288312d4f0 (127.4.59.2:40301): tablet 7e096348d2dc44d28e89d125c949d37d (table test-table [id=fe5779a710be4137ba2ae6bce91a6cce]) successfully deleted
W20260620 09:22:05.256479 12374 tablet.cc:2369] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:22:05.286415 12409 raft_consensus.cc:482] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:22:05.286512 12409 raft_consensus.cc:504] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.286840 12409 leader_election.cc:288] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6952e6c79a6475fb07747c4a4ace93e (127.4.59.1:39897), c09e2e49b59d443c8d7b25cce0229255 (127.4.59.3:35377)
I20260620 09:22:05.290225 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.287254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:35255 (local address 127.4.59.3:35377)
0620 09:22:05.287309 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.287311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.287320 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:22:05.287375 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.287376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.287401 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.287454 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:22:05.287669 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.288379 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.289465 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.289887 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.289960 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.289970 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.290118 (+ 148us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.290155 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 09:22:05.290242 12430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.287139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:35377 (local address 127.4.59.2:35255)
0620 09:22:05.287246 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:05.287256 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:05.287309 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:05.287469 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:05.287470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:05.287471 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:22:05.287554 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.287555 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.288499 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.288500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.289376 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.289378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.289969 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.289970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.290042 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.290047 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 09:22:05.290104 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 09:22:05.290140 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":2,"thread_start_us":58,"threads_started":1}
I20260620 09:22:05.290540 12311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f773b13495c14bc8818acad133714d3a" candidate_uuid: "39c8f9995e714f729ff000288312d4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c09e2e49b59d443c8d7b25cce0229255" is_pre_election: true
I20260620 09:22:05.290642 12126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.287016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:39897 (local address 127.4.59.2:34815)
0620 09:22:05.287077 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:05.287089 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:05.287136 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:05.287419 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:05.287419 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:05.287421 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:22:05.287496 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.287498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.288465 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.288466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.289673 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.289675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.290402 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.290403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.290478 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.290482 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:22:05.290527 (+ 45us) client_negotiation.cc:240] Negotiation successful
0620 09:22:05.290577 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 09:22:05.290668 12311 raft_consensus.cc:2454] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39c8f9995e714f729ff000288312d4f0 in term 0.
I20260620 09:22:05.290855 12400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.287225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:34815 (local address 127.4.59.1:39897)
0620 09:22:05.287270 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.287272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.287280 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:22:05.287307 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.287308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.287319 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.287369 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:22:05.287578 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.288390 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.289772 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.290282 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.290364 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.290376 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.290724 (+ 348us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.290775 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 09:22:05.290848 12118 leader_election.cc:302] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39c8f9995e714f729ff000288312d4f0, c09e2e49b59d443c8d7b25cce0229255; no voters:
I20260620 09:22:05.290983 12409 raft_consensus.cc:2781] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:22:05.291021 12409 raft_consensus.cc:482] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:22:05.291050 12409 raft_consensus.cc:3037] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:05.291064 12047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f773b13495c14bc8818acad133714d3a" candidate_uuid: "39c8f9995e714f729ff000288312d4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6952e6c79a6475fb07747c4a4ace93e" is_pre_election: true
I20260620 09:22:05.291146 12047 raft_consensus.cc:2454] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39c8f9995e714f729ff000288312d4f0 in term 0.
I20260620 09:22:05.291777 12409 raft_consensus.cc:504] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.291914 12409 leader_election.cc:288] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [CANDIDATE]: Term 1 election: Requested vote from peers d6952e6c79a6475fb07747c4a4ace93e (127.4.59.1:39897), c09e2e49b59d443c8d7b25cce0229255 (127.4.59.3:35377)
I20260620 09:22:05.292066 12311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f773b13495c14bc8818acad133714d3a" candidate_uuid: "39c8f9995e714f729ff000288312d4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c09e2e49b59d443c8d7b25cce0229255"
I20260620 09:22:05.292099 12047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f773b13495c14bc8818acad133714d3a" candidate_uuid: "39c8f9995e714f729ff000288312d4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6952e6c79a6475fb07747c4a4ace93e"
I20260620 09:22:05.292156 12311 raft_consensus.cc:3037] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:05.292171 12047 raft_consensus.cc:3037] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:05.292917 12311 raft_consensus.cc:2454] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39c8f9995e714f729ff000288312d4f0 in term 1.
I20260620 09:22:05.293185 12118 leader_election.cc:302] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39c8f9995e714f729ff000288312d4f0, c09e2e49b59d443c8d7b25cce0229255; no voters:
I20260620 09:22:05.293262 12409 raft_consensus.cc:2781] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:22:05.293303 12047 raft_consensus.cc:2454] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39c8f9995e714f729ff000288312d4f0 in term 1.
I20260620 09:22:05.293401 12409 raft_consensus.cc:686] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [term 1 LEADER]: Becoming Leader. State: Replica: 39c8f9995e714f729ff000288312d4f0, State: Running, Role: LEADER
I20260620 09:22:05.293478 12409 consensus_queue.cc:237] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.294166 11890 catalog_manager.cc:5495] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39c8f9995e714f729ff000288312d4f0 (127.4.59.2). New cstate: current_term: 1 leader_uuid: "39c8f9995e714f729ff000288312d4f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } health_report { overall_health: UNKNOWN } } }
Jun 20 09:22:05 dist-test-slave-414w krb5kdc[11845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947324, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.62@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM
I20260620 09:22:05.337463 12438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.329913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35907 (local address 127.0.0.1:39880)
0620 09:22:05.330112 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:05.330122 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:05.330200 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:05.330339 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:05.330340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:05.330635 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:05.330682 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.330685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.331371 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.331372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.331884 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.331885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.332273 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.332274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.332338 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.333183 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:05.333189 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:05.334725 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:05.336326 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:05.336332 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:05.336334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:05.336521 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:05.336732 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:05.336732 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:05.336733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:05.336795 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:05.337037 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:05.337043 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:05.337089 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 09:22:05.337209 (+ 120us) client_negotiation.cc:240] Negotiation successful
0620 09:22:05.337334 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":102,"threads_started":1}
I20260620 09:22:05.337467 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.329973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39880 (local address 127.4.59.62:35907)
0620 09:22:05.330049 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.330051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.330181 (+ 130us) server_negotiation.cc:407] Connection header received
0620 09:22:05.330273 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.330273 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.330284 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.330335 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:05.330767 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.331264 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.331979 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.332208 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.332264 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.334818 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:05.334819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:05.334820 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:05.334827 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:05.336207 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:05.336609 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:05.336610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:05.336611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:05.336666 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:05.336859 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:05.336860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:05.336861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:05.336970 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:05.337046 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.337251 (+ 205us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.337384 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":48}
I20260620 09:22:05.339666 11890 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:39880:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 09:22:05.339876 11890 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:22:05.341339 12305 tablet_service.cc:1467] Processing CreateTablet for tablet 4d23d946622c4fb3ac017a3182d4b272 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=efaceb4e65eb45229cc27c300e155795]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 09:22:05.341385 12167 tablet_service.cc:1467] Processing CreateTablet for tablet 4d23d946622c4fb3ac017a3182d4b272 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=efaceb4e65eb45229cc27c300e155795]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 09:22:05.341503 12305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d23d946622c4fb3ac017a3182d4b272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:05.341518 12167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d23d946622c4fb3ac017a3182d4b272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:05.342967 12402 tablet_bootstrap.cc:492] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0: Bootstrap starting.
I20260620 09:22:05.343065 12027 tablet_service.cc:1467] Processing CreateTablet for tablet 4d23d946622c4fb3ac017a3182d4b272 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=efaceb4e65eb45229cc27c300e155795]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 09:22:05.343190 12027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d23d946622c4fb3ac017a3182d4b272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:05.343392 12402 tablet_bootstrap.cc:654] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:05.343717 12403 tablet_bootstrap.cc:492] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255: Bootstrap starting.
I20260620 09:22:05.344101 12403 tablet_bootstrap.cc:654] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:05.344309 12401 tablet_bootstrap.cc:492] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e: Bootstrap starting.
I20260620 09:22:05.344688 12402 tablet_bootstrap.cc:492] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0: No bootstrap required, opened a new log
I20260620 09:22:05.344707 12401 tablet_bootstrap.cc:654] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:05.344738 12402 ts_tablet_manager.cc:1397] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:22:05.344880 12402 raft_consensus.cc:348] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.344944 12402 raft_consensus.cc:374] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:05.345295 12403 tablet_bootstrap.cc:492] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255: No bootstrap required, opened a new log
I20260620 09:22:05.345340 12403 ts_tablet_manager.cc:1397] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:22:05.345417 12401 tablet_bootstrap.cc:492] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e: No bootstrap required, opened a new log
I20260620 09:22:05.345450 12401 ts_tablet_manager.cc:1397] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 09:22:05.345472 12403 raft_consensus.cc:348] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.345535 12403 raft_consensus.cc:374] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:05.345553 12403 raft_consensus.cc:729] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c09e2e49b59d443c8d7b25cce0229255, State: Initialized, Role: FOLLOWER
I20260620 09:22:05.345551 12401 raft_consensus.cc:348] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.345598 12403 consensus_queue.cc:260] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.345629 12401 raft_consensus.cc:374] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:05.345652 12401 raft_consensus.cc:729] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6952e6c79a6475fb07747c4a4ace93e, State: Initialized, Role: FOLLOWER
I20260620 09:22:05.345682 12403 ts_tablet_manager.cc:1428] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:05.345690 12401 consensus_queue.cc:260] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.345777 12421 tablet_replica.cc:439] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } }
I20260620 09:22:05.345813 12401 ts_tablet_manager.cc:1428] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 09:22:05.345831 12421 tablet_replica.cc:442] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:22:05.345942 12408 tablet_replica.cc:439] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } }
I20260620 09:22:05.346022 12408 tablet_replica.cc:442] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:22:05.344964 12402 raft_consensus.cc:729] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39c8f9995e714f729ff000288312d4f0, State: Initialized, Role: FOLLOWER
I20260620 09:22:05.346160 12402 consensus_queue.cc:260] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.346256 12402 ts_tablet_manager.cc:1428] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 09:22:05.346520 12431 tablet_replica.cc:439] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } }
I20260620 09:22:05.346575 12431 tablet_replica.cc:442] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:22:05.607615 12421 raft_consensus.cc:482] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:22:05.607720 12421 raft_consensus.cc:504] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.607906 12421 leader_election.cc:288] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6952e6c79a6475fb07747c4a4ace93e (127.4.59.1:39897), 39c8f9995e714f729ff000288312d4f0 (127.4.59.2:40301)
I20260620 09:22:05.608222 12047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d23d946622c4fb3ac017a3182d4b272" candidate_uuid: "c09e2e49b59d443c8d7b25cce0229255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6952e6c79a6475fb07747c4a4ace93e" is_pre_election: true
I20260620 09:22:05.608212 12187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d23d946622c4fb3ac017a3182d4b272" candidate_uuid: "c09e2e49b59d443c8d7b25cce0229255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c8f9995e714f729ff000288312d4f0" is_pre_election: true
I20260620 09:22:05.608328 12047 raft_consensus.cc:2454] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c09e2e49b59d443c8d7b25cce0229255 in term 0.
I20260620 09:22:05.608328 12187 raft_consensus.cc:2454] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c09e2e49b59d443c8d7b25cce0229255 in term 0.
I20260620 09:22:05.608469 12258 leader_election.cc:302] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c09e2e49b59d443c8d7b25cce0229255, d6952e6c79a6475fb07747c4a4ace93e; no voters:
I20260620 09:22:05.608589 12421 raft_consensus.cc:2781] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:22:05.608631 12421 raft_consensus.cc:482] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:22:05.608655 12421 raft_consensus.cc:3037] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:05.609534 12421 raft_consensus.cc:504] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.609675 12421 leader_election.cc:288] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [CANDIDATE]: Term 1 election: Requested vote from peers d6952e6c79a6475fb07747c4a4ace93e (127.4.59.1:39897), 39c8f9995e714f729ff000288312d4f0 (127.4.59.2:40301)
I20260620 09:22:05.609838 12187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d23d946622c4fb3ac017a3182d4b272" candidate_uuid: "c09e2e49b59d443c8d7b25cce0229255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c8f9995e714f729ff000288312d4f0"
I20260620 09:22:05.609858 12047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d23d946622c4fb3ac017a3182d4b272" candidate_uuid: "c09e2e49b59d443c8d7b25cce0229255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6952e6c79a6475fb07747c4a4ace93e"
I20260620 09:22:05.609946 12187 raft_consensus.cc:3037] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:05.609947 12047 raft_consensus.cc:3037] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:05.610607 12187 raft_consensus.cc:2454] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c09e2e49b59d443c8d7b25cce0229255 in term 1.
I20260620 09:22:05.610740 12047 raft_consensus.cc:2454] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c09e2e49b59d443c8d7b25cce0229255 in term 1.
I20260620 09:22:05.610785 12261 leader_election.cc:302] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39c8f9995e714f729ff000288312d4f0, c09e2e49b59d443c8d7b25cce0229255; no voters:
I20260620 09:22:05.610955 12421 raft_consensus.cc:2781] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:22:05.611043 12421 raft_consensus.cc:686] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [term 1 LEADER]: Becoming Leader. State: Replica: c09e2e49b59d443c8d7b25cce0229255, State: Running, Role: LEADER
I20260620 09:22:05.611107 12421 consensus_queue.cc:237] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.611397 12411 tablet_replica.cc:439] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255: TxnStatusTablet state changed. Reason: New leader c09e2e49b59d443c8d7b25cce0229255. Latest consensus state: current_term: 1 leader_uuid: "c09e2e49b59d443c8d7b25cce0229255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } }
I20260620 09:22:05.611480 12411 tablet_replica.cc:442] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255: This TxnStatusTablet replica's current role is: LEADER
I20260620 09:22:05.611632 12443 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 09:22:05.611788 12443 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 09:22:05.611739 11890 catalog_manager.cc:5495] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 reported cstate change: term changed from 0 to 1, leader changed from <none> to c09e2e49b59d443c8d7b25cce0229255 (127.4.59.3). New cstate: current_term: 1 leader_uuid: "c09e2e49b59d443c8d7b25cce0229255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } health_report { overall_health: HEALTHY } } }
I20260620 09:22:05.633976 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.630635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36130 (local address 127.4.59.3:35377)
0620 09:22:05.630680 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.630682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.630712 (+ 30us) server_negotiation.cc:407] Connection header received
0620 09:22:05.630797 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.630798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.630809 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.630889 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:22:05.631030 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.631799 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.633020 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.633348 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.633422 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.633701 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:22:05.633743 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:22:05.633810 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.633890 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.633911 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 09:22:05.634028 12438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.630576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:35377 (local address 127.0.0.1:36130)
0620 09:22:05.630665 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:05.630676 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:05.630729 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:05.630898 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:05.630899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:05.630900 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 09:22:05.630955 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.630958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.631927 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.631939 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.632920 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.632924 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.633445 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.633446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.633532 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.633550 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 09:22:05.633821 (+ 271us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 09:22:05.633832 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 09:22:05.633889 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 09:22:05.633944 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260620 09:22:05.634985 12047 raft_consensus.cc:1261] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Refusing update from remote peer c09e2e49b59d443c8d7b25cce0229255: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:22:05.635011 12187 raft_consensus.cc:1261] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0 [term 1 FOLLOWER]: Refusing update from remote peer c09e2e49b59d443c8d7b25cce0229255: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:22:05.635197 12421 consensus_queue.cc:1035] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:22:05.635625 12421 consensus_queue.cc:1035] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:22:05.636045 12408 tablet_replica.cc:439] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e: TxnStatusTablet state changed. Reason: New leader c09e2e49b59d443c8d7b25cce0229255. Latest consensus state: current_term: 1 leader_uuid: "c09e2e49b59d443c8d7b25cce0229255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } }
I20260620 09:22:05.636147 12408 tablet_replica.cc:442] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:22:05.636360 12431 tablet_replica.cc:439] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0: TxnStatusTablet state changed. Reason: New leader c09e2e49b59d443c8d7b25cce0229255. Latest consensus state: current_term: 1 leader_uuid: "c09e2e49b59d443c8d7b25cce0229255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } }
I20260620 09:22:05.636459 12431 tablet_replica.cc:442] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:22:05.636883 12421 tablet_replica.cc:439] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c09e2e49b59d443c8d7b25cce0229255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } }
I20260620 09:22:05.636970 12421 tablet_replica.cc:442] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255: This TxnStatusTablet replica's current role is: LEADER
I20260620 09:22:05.637086 12414 mvcc.cc:205] Tried to move back new op lower bound from 7298856245799133184 to 7298856245703512064. Current Snapshot: MvccSnapshot[applied={T|T < 7298856245799133184 or (T in {7298856245799133184})}]
I20260620 09:22:05.637220 12407 tablet_replica.cc:439] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c09e2e49b59d443c8d7b25cce0229255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } }
I20260620 09:22:05.637288 12407 tablet_replica.cc:442] T 4d23d946622c4fb3ac017a3182d4b272 P c09e2e49b59d443c8d7b25cce0229255: This TxnStatusTablet replica's current role is: LEADER
I20260620 09:22:05.637455 12415 mvcc.cc:205] Tried to move back new op lower bound from 7298856245799133184 to 7298856245703512064. Current Snapshot: MvccSnapshot[applied={T|T < 7298856245799133184}]
I20260620 09:22:05.637814 12431 tablet_replica.cc:439] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c09e2e49b59d443c8d7b25cce0229255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } }
I20260620 09:22:05.637887 12431 tablet_replica.cc:442] T 4d23d946622c4fb3ac017a3182d4b272 P 39c8f9995e714f729ff000288312d4f0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:22:05.638060 12408 tablet_replica.cc:439] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c09e2e49b59d443c8d7b25cce0229255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } } }
I20260620 09:22:05.638141 12408 tablet_replica.cc:442] T 4d23d946622c4fb3ac017a3182d4b272 P d6952e6c79a6475fb07747c4a4ace93e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 09:22:05.641345 12397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.638515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39218 (local address 127.4.59.2:40301)
0620 09:22:05.638571 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.638573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.638581 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:22:05.638617 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.638618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.638638 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.638706 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:22:05.638875 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.639529 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.640483 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.640704 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.640793 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.640958 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:22:05.641066 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:22:05.641142 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.641236 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.641258 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 09:22:05.647003 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.643979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:54631 (local address 127.4.59.3:35377)
0620 09:22:05.644020 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.644023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.644048 (+ 25us) server_negotiation.cc:407] Connection header received
0620 09:22:05.644120 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.644121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.644131 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.644175 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:22:05.644344 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.645180 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.646166 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.646455 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.646520 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.646733 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:22:05.646774 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:22:05.646828 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.646915 (+ 87us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.646932 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:22:05.647028 12450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.643847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:35377 (local address 127.4.59.2:54631)
0620 09:22:05.643994 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:05.644006 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:05.644064 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:05.644187 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:05.644188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:05.644190 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 09:22:05.644254 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.644256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.645246 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.645246 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.646072 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.646075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.646545 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.646546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.646617 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.646632 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 09:22:05.646856 (+ 224us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 09:22:05.646858 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:22:05.646904 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 09:22:05.646948 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260620 09:22:05.649394 12047 raft_consensus.cc:1261] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Refusing update from remote peer 39c8f9995e714f729ff000288312d4f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:22:05.649418 12311 raft_consensus.cc:1261] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Refusing update from remote peer 39c8f9995e714f729ff000288312d4f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:22:05.649681 12431 consensus_queue.cc:1035] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:22:05.649832 12409 consensus_queue.cc:1035] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:22:05.658484 12455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.655815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:40301 (local address 127.4.59.3:35623)
0620 09:22:05.655962 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:05.655975 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:05.656041 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:05.656188 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:05.656189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:05.656191 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 09:22:05.656238 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.656239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.656869 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.656870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.657667 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:05.657669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.658067 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.658068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:05.658127 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.658143 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 09:22:05.658316 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 09:22:05.658318 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:22:05.658369 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 09:22:05.658414 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260620 09:22:05.658468 12397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.655852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:35623 (local address 127.4.59.2:40301)
0620 09:22:05.655892 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.655895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.656027 (+ 132us) server_negotiation.cc:407] Connection header received
0620 09:22:05.656097 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.656097 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.656107 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.656170 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:22:05.656317 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.656787 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.657743 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.657984 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.658056 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.658221 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:22:05.658254 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:22:05.658306 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.658372 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.658391 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 09:22:05.673528 11888 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:22:05.673631 11888 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:22:05.674249 11888 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd: Sending DeleteTablet for 3 replicas of tablet f773b13495c14bc8818acad133714d3a
I20260620 09:22:05.674540 12305 tablet_service.cc:1514] Processing DeleteTablet for tablet f773b13495c14bc8818acad133714d3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:22:05 UTC) from {username='oryx', principal='oryx/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:36110
I20260620 09:22:05.675052 12425 tablet_replica.cc:330] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255: stopping tablet replica
I20260620 09:22:05.675144 12425 raft_consensus.cc:2229] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:22:05.675180 12425 raft_consensus.cc:2258] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:22:05.675246 12027 tablet_service.cc:1514] Processing DeleteTablet for tablet f773b13495c14bc8818acad133714d3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:22:05 UTC) from {username='oryx', principal='oryx/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:43212
I20260620 09:22:05.675340 12167 tablet_service.cc:1514] Processing DeleteTablet for tablet f773b13495c14bc8818acad133714d3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:22:05 UTC) from {username='oryx', principal='oryx/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:39216
I20260620 09:22:05.675340 12424 tablet_replica.cc:330] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e: stopping tablet replica
I20260620 09:22:05.675402 12424 raft_consensus.cc:2229] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:22:05.675441 12424 raft_consensus.cc:2258] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:22:05.675490 12426 tablet_replica.cc:330] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0: stopping tablet replica
I20260620 09:22:05.675539 12426 raft_consensus.cc:2229] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:22:05.675612 12426 raft_consensus.cc:2258] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:22:05.675907 12424 ts_tablet_manager.cc:1905] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:22:05.676025 12425 ts_tablet_manager.cc:1905] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:22:05.677735 12424 ts_tablet_manager.cc:1918] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 09:22:05.677781 12426 ts_tablet_manager.cc:1905] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:22:05.677807 12424 log.cc:1196] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/wal/wals/f773b13495c14bc8818acad133714d3a
I20260620 09:22:05.678030 12424 ts_tablet_manager.cc:1939] T f773b13495c14bc8818acad133714d3a P d6952e6c79a6475fb07747c4a4ace93e: Deleting consensus metadata
I20260620 09:22:05.678092 12425 ts_tablet_manager.cc:1918] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 09:22:05.678154 12425 log.cc:1196] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/f773b13495c14bc8818acad133714d3a
I20260620 09:22:05.678364 12425 ts_tablet_manager.cc:1939] T f773b13495c14bc8818acad133714d3a P c09e2e49b59d443c8d7b25cce0229255: Deleting consensus metadata
I20260620 09:22:05.678661 11874 catalog_manager.cc:4841] TS c09e2e49b59d443c8d7b25cce0229255 (127.4.59.3:35377): tablet f773b13495c14bc8818acad133714d3a (table test-table [id=6ab743972ed246ce9ccdaa3fc4694605]) successfully deleted
I20260620 09:22:05.678984 11874 catalog_manager.cc:4841] TS d6952e6c79a6475fb07747c4a4ace93e (127.4.59.1:39897): tablet f773b13495c14bc8818acad133714d3a (table test-table [id=6ab743972ed246ce9ccdaa3fc4694605]) successfully deleted
I20260620 09:22:05.679241 12426 ts_tablet_manager.cc:1918] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 09:22:05.679306 12426 log.cc:1196] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/f773b13495c14bc8818acad133714d3a
I20260620 09:22:05.679507 12426 ts_tablet_manager.cc:1939] T f773b13495c14bc8818acad133714d3a P 39c8f9995e714f729ff000288312d4f0: Deleting consensus metadata
I20260620 09:22:05.679788 11877 catalog_manager.cc:4841] TS 39c8f9995e714f729ff000288312d4f0 (127.4.59.2:40301): tablet f773b13495c14bc8818acad133714d3a (table test-table [id=6ab743972ed246ce9ccdaa3fc4694605]) successfully deleted
Jun 20 09:22:05 dist-test-slave-414w krb5kdc[11845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 09:22:05 dist-test-slave-414w krb5kdc[11845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:05 dist-test-slave-414w krb5kdc[11845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
W20260620 09:22:05.720181 11940 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:05.713432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39896 (local address 127.4.59.62:35907)
0620 09:22:05.713481 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.713483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.713623 (+ 140us) server_negotiation.cc:407] Connection header received
0620 09:22:05.713701 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.713702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.713714 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.713772 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:05.714484 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.715256 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.716345 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.716653 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.716725 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.720100 (+ 3375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39896: BlockingRecv error: recv got EOF from 127.0.0.1:39896 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 20 09:22:05 dist-test-slave-414w krb5kdc[11845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM
I20260620 09:22:05.730486 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.720953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39900 (local address 127.4.59.62:35907)
0620 09:22:05.720989 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.720991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.721258 (+ 267us) server_negotiation.cc:407] Connection header received
0620 09:22:05.721318 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.721319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.721330 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.721390 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:05.721974 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.722713 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.723680 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.723974 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.724043 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.727447 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:05.727464 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:05.727465 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:05.727472 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:05.729265 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:05.729648 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:05.729650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:05.729650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:05.729709 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:05.729930 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:05.729931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:05.729931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:05.730062 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:05.730168 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.730307 (+ 139us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.730384 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:22:05.732438 11888 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:22:05.732625 11888 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:22:05.734095 12167 tablet_service.cc:1467] Processing CreateTablet for tablet 71f95e1e161b44e1a7db83e863a30ca0 (DEFAULT_TABLE table=test-table [id=5f53fe8d7a374d149f6d38105c7becbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:22:05.734166 12027 tablet_service.cc:1467] Processing CreateTablet for tablet 71f95e1e161b44e1a7db83e863a30ca0 (DEFAULT_TABLE table=test-table [id=5f53fe8d7a374d149f6d38105c7becbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:22:05.734236 12167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71f95e1e161b44e1a7db83e863a30ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:05.734292 12027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71f95e1e161b44e1a7db83e863a30ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:05.735589 12305 tablet_service.cc:1467] Processing CreateTablet for tablet 71f95e1e161b44e1a7db83e863a30ca0 (DEFAULT_TABLE table=test-table [id=5f53fe8d7a374d149f6d38105c7becbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:22:05.735639 12401 tablet_bootstrap.cc:492] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e: Bootstrap starting.
I20260620 09:22:05.735699 12305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71f95e1e161b44e1a7db83e863a30ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:05.736034 12401 tablet_bootstrap.cc:654] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:05.736343 12402 tablet_bootstrap.cc:492] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0: Bootstrap starting.
I20260620 09:22:05.736732 12402 tablet_bootstrap.cc:654] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:05.736922 12403 tablet_bootstrap.cc:492] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255: Bootstrap starting.
I20260620 09:22:05.737644 12403 tablet_bootstrap.cc:654] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:05.737715 12402 tablet_bootstrap.cc:492] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0: No bootstrap required, opened a new log
I20260620 09:22:05.737758 12402 ts_tablet_manager.cc:1397] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 09:22:05.737901 12402 raft_consensus.cc:348] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.737977 12402 raft_consensus.cc:374] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:05.737998 12402 raft_consensus.cc:729] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39c8f9995e714f729ff000288312d4f0, State: Initialized, Role: FOLLOWER
I20260620 09:22:05.738039 12402 consensus_queue.cc:260] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.738122 12402 ts_tablet_manager.cc:1428] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 09:22:05.738531 12401 tablet_bootstrap.cc:492] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e: No bootstrap required, opened a new log
I20260620 09:22:05.738582 12401 ts_tablet_manager.cc:1397] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 09:22:05.738732 12401 raft_consensus.cc:348] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.738812 12401 raft_consensus.cc:374] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:05.738839 12401 raft_consensus.cc:729] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6952e6c79a6475fb07747c4a4ace93e, State: Initialized, Role: FOLLOWER
I20260620 09:22:05.738880 12401 consensus_queue.cc:260] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.738971 12401 ts_tablet_manager.cc:1428] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:05.739415 12403 tablet_bootstrap.cc:492] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255: No bootstrap required, opened a new log
I20260620 09:22:05.739462 12403 ts_tablet_manager.cc:1397] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260620 09:22:05.739559 12403 raft_consensus.cc:348] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.739616 12403 raft_consensus.cc:374] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:05.739636 12403 raft_consensus.cc:729] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c09e2e49b59d443c8d7b25cce0229255, State: Initialized, Role: FOLLOWER
I20260620 09:22:05.739674 12403 consensus_queue.cc:260] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.739758 12403 ts_tablet_manager.cc:1428] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:05.922286 12431 raft_consensus.cc:482] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:22:05.922390 12431 raft_consensus.cc:504] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.922549 12431 leader_election.cc:288] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6952e6c79a6475fb07747c4a4ace93e (127.4.59.1:39897), c09e2e49b59d443c8d7b25cce0229255 (127.4.59.3:35377)
I20260620 09:22:05.922750 12047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71f95e1e161b44e1a7db83e863a30ca0" candidate_uuid: "39c8f9995e714f729ff000288312d4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6952e6c79a6475fb07747c4a4ace93e" is_pre_election: true
I20260620 09:22:05.922828 12311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71f95e1e161b44e1a7db83e863a30ca0" candidate_uuid: "39c8f9995e714f729ff000288312d4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c09e2e49b59d443c8d7b25cce0229255" is_pre_election: true
I20260620 09:22:05.922881 12047 raft_consensus.cc:2454] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39c8f9995e714f729ff000288312d4f0 in term 0.
I20260620 09:22:05.922911 12311 raft_consensus.cc:2454] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39c8f9995e714f729ff000288312d4f0 in term 0.
I20260620 09:22:05.923089 12118 leader_election.cc:302] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39c8f9995e714f729ff000288312d4f0, d6952e6c79a6475fb07747c4a4ace93e; no voters:
I20260620 09:22:05.923180 12431 raft_consensus.cc:2781] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:22:05.923211 12431 raft_consensus.cc:482] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:22:05.923239 12431 raft_consensus.cc:3037] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:05.923945 12431 raft_consensus.cc:504] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.924069 12431 leader_election.cc:288] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [CANDIDATE]: Term 1 election: Requested vote from peers d6952e6c79a6475fb07747c4a4ace93e (127.4.59.1:39897), c09e2e49b59d443c8d7b25cce0229255 (127.4.59.3:35377)
I20260620 09:22:05.924244 12311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71f95e1e161b44e1a7db83e863a30ca0" candidate_uuid: "39c8f9995e714f729ff000288312d4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c09e2e49b59d443c8d7b25cce0229255"
I20260620 09:22:05.924240 12047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71f95e1e161b44e1a7db83e863a30ca0" candidate_uuid: "39c8f9995e714f729ff000288312d4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6952e6c79a6475fb07747c4a4ace93e"
I20260620 09:22:05.924310 12311 raft_consensus.cc:3037] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:05.924412 12047 raft_consensus.cc:3037] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:05.924950 12311 raft_consensus.cc:2454] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39c8f9995e714f729ff000288312d4f0 in term 1.
I20260620 09:22:05.925112 12118 leader_election.cc:302] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39c8f9995e714f729ff000288312d4f0, c09e2e49b59d443c8d7b25cce0229255; no voters:
I20260620 09:22:05.925184 12047 raft_consensus.cc:2454] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39c8f9995e714f729ff000288312d4f0 in term 1.
I20260620 09:22:05.925225 12431 raft_consensus.cc:2781] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:22:05.925283 12431 raft_consensus.cc:686] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [term 1 LEADER]: Becoming Leader. State: Replica: 39c8f9995e714f729ff000288312d4f0, State: Running, Role: LEADER
I20260620 09:22:05.925333 12431 consensus_queue.cc:237] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.925863 11888 catalog_manager.cc:5495] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39c8f9995e714f729ff000288312d4f0 (127.4.59.2). New cstate: current_term: 1 leader_uuid: "39c8f9995e714f729ff000288312d4f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } health_report { overall_health: UNKNOWN } } }
I20260620 09:22:05.969105 12397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:05.966859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39234 (local address 127.4.59.2:40301)
0620 09:22:05.966902 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:05.966905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:05.966962 (+ 57us) server_negotiation.cc:407] Connection header received
0620 09:22:05.967026 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:05.967026 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:05.967035 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:05.967092 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:22:05.967234 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.967754 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.968513 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:05.968724 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:05.968784 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:05.968907 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:22:05.968933 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:22:05.968975 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:05.969026 (+ 51us) server_negotiation.cc:299] Negotiation successful
0620 09:22:05.969041 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:22:05.969975 12311 raft_consensus.cc:1261] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Refusing update from remote peer 39c8f9995e714f729ff000288312d4f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:22:05.969977 12047 raft_consensus.cc:1261] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Refusing update from remote peer 39c8f9995e714f729ff000288312d4f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:22:05.970221 12409 consensus_queue.cc:1035] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:22:05.970355 12431 consensus_queue.cc:1035] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:22:05.973392 11889 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:22:05.973492 11889 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:22:05.974040 11889 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd: Sending DeleteTablet for 3 replicas of tablet 71f95e1e161b44e1a7db83e863a30ca0
I20260620 09:22:05.974272 12027 tablet_service.cc:1514] Processing DeleteTablet for tablet 71f95e1e161b44e1a7db83e863a30ca0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:22:05 UTC) from {username='oryx', principal='oryx/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:43212
I20260620 09:22:05.974270 12167 tablet_service.cc:1514] Processing DeleteTablet for tablet 71f95e1e161b44e1a7db83e863a30ca0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:22:05 UTC) from {username='oryx', principal='oryx/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:39216
I20260620 09:22:05.974300 12305 tablet_service.cc:1514] Processing DeleteTablet for tablet 71f95e1e161b44e1a7db83e863a30ca0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:22:05 UTC) from {username='oryx', principal='oryx/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:36110
I20260620 09:22:05.974362 12424 tablet_replica.cc:330] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e: stopping tablet replica
I20260620 09:22:05.974387 12425 tablet_replica.cc:330] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255: stopping tablet replica
I20260620 09:22:05.974416 12424 raft_consensus.cc:2229] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:22:05.974450 12425 raft_consensus.cc:2229] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:22:05.974468 12424 raft_consensus.cc:2258] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:22:05.974498 12425 raft_consensus.cc:2258] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:22:05.974541 12426 tablet_replica.cc:330] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0: stopping tablet replica
I20260620 09:22:05.974489 11889 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 09:22:05.974593 12426 raft_consensus.cc:2229] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [term 1 LEADER]: Raft consensus shutting down.
W20260620 09:22:05.974615 11889 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:22:05.974643 12426 raft_consensus.cc:2258] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:22:05.974845 12425 ts_tablet_manager.cc:1905] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:22:05.974973 12424 ts_tablet_manager.cc:1905] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:22:05.976230 12305 tablet_service.cc:1467] Processing CreateTablet for tablet 4b85684adaa142fba69deae1dd455697 (DEFAULT_TABLE table=test-table [id=3f88d0e5cc894b68aa1e978aeaf2218e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:22:05.976269 12425 ts_tablet_manager.cc:1918] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:22:05.976328 12425 log.cc:1196] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/71f95e1e161b44e1a7db83e863a30ca0
I20260620 09:22:05.976349 12305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b85684adaa142fba69deae1dd455697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:05.976415 12426 ts_tablet_manager.cc:1905] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:22:05.976552 12425 ts_tablet_manager.cc:1939] T 71f95e1e161b44e1a7db83e863a30ca0 P c09e2e49b59d443c8d7b25cce0229255: Deleting consensus metadata
I20260620 09:22:05.976541 12424 ts_tablet_manager.cc:1918] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:22:05.976589 12424 log.cc:1196] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-0/wal/wals/71f95e1e161b44e1a7db83e863a30ca0
I20260620 09:22:05.976804 12424 ts_tablet_manager.cc:1939] T 71f95e1e161b44e1a7db83e863a30ca0 P d6952e6c79a6475fb07747c4a4ace93e: Deleting consensus metadata
I20260620 09:22:05.977555 12403 tablet_bootstrap.cc:492] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255: Bootstrap starting.
I20260620 09:22:05.977614 11874 catalog_manager.cc:4841] TS d6952e6c79a6475fb07747c4a4ace93e (127.4.59.1:39897): tablet 71f95e1e161b44e1a7db83e863a30ca0 (table test-table [id=5f53fe8d7a374d149f6d38105c7becbe]) successfully deleted
I20260620 09:22:05.977689 11874 catalog_manager.cc:4841] TS c09e2e49b59d443c8d7b25cce0229255 (127.4.59.3:35377): tablet 71f95e1e161b44e1a7db83e863a30ca0 (table test-table [id=5f53fe8d7a374d149f6d38105c7becbe]) successfully deleted
I20260620 09:22:05.977806 12027 tablet_service.cc:1467] Processing CreateTablet for tablet 4b85684adaa142fba69deae1dd455697 (DEFAULT_TABLE table=test-table [id=3f88d0e5cc894b68aa1e978aeaf2218e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:22:05.977849 12403 tablet_bootstrap.cc:654] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:05.977900 12027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b85684adaa142fba69deae1dd455697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:05.977907 12167 tablet_service.cc:1467] Processing CreateTablet for tablet 4b85684adaa142fba69deae1dd455697 (DEFAULT_TABLE table=test-table [id=3f88d0e5cc894b68aa1e978aeaf2218e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:22:05.977988 12167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b85684adaa142fba69deae1dd455697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:05.978196 12426 ts_tablet_manager.cc:1918] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:22:05.978235 12426 log.cc:1196] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/71f95e1e161b44e1a7db83e863a30ca0
I20260620 09:22:05.978399 12426 ts_tablet_manager.cc:1939] T 71f95e1e161b44e1a7db83e863a30ca0 P 39c8f9995e714f729ff000288312d4f0: Deleting consensus metadata
I20260620 09:22:05.978823 11877 catalog_manager.cc:4841] TS 39c8f9995e714f729ff000288312d4f0 (127.4.59.2:40301): tablet 71f95e1e161b44e1a7db83e863a30ca0 (table test-table [id=5f53fe8d7a374d149f6d38105c7becbe]) successfully deleted
I20260620 09:22:05.979030 12401 tablet_bootstrap.cc:492] T 4b85684adaa142fba69deae1dd455697 P d6952e6c79a6475fb07747c4a4ace93e: Bootstrap starting.
I20260620 09:22:05.979079 12402 tablet_bootstrap.cc:492] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0: Bootstrap starting.
I20260620 09:22:05.979365 12402 tablet_bootstrap.cc:654] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:05.979385 12403 tablet_bootstrap.cc:492] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255: No bootstrap required, opened a new log
I20260620 09:22:05.979431 12403 ts_tablet_manager.cc:1397] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 09:22:05.979542 12403 raft_consensus.cc:348] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.979604 12403 raft_consensus.cc:374] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:05.979624 12403 raft_consensus.cc:729] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c09e2e49b59d443c8d7b25cce0229255, State: Initialized, Role: FOLLOWER
I20260620 09:22:05.979668 12403 consensus_queue.cc:260] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.979740 12401 tablet_bootstrap.cc:654] T 4b85684adaa142fba69deae1dd455697 P d6952e6c79a6475fb07747c4a4ace93e: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:05.979770 12403 ts_tablet_manager.cc:1428] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260620 09:22:05.979844 12402 tablet_bootstrap.cc:492] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0: No bootstrap required, opened a new log
I20260620 09:22:05.979887 12402 ts_tablet_manager.cc:1397] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 09:22:05.979961 12402 raft_consensus.cc:348] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.980001 12402 raft_consensus.cc:374] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:05.980014 12402 raft_consensus.cc:729] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39c8f9995e714f729ff000288312d4f0, State: Initialized, Role: FOLLOWER
I20260620 09:22:05.980069 12402 consensus_queue.cc:260] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.980172 12402 ts_tablet_manager.cc:1428] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:05.980341 12401 tablet_bootstrap.cc:492] T 4b85684adaa142fba69deae1dd455697 P d6952e6c79a6475fb07747c4a4ace93e: No bootstrap required, opened a new log
I20260620 09:22:05.980381 12401 ts_tablet_manager.cc:1397] T 4b85684adaa142fba69deae1dd455697 P d6952e6c79a6475fb07747c4a4ace93e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 09:22:05.980505 12401 raft_consensus.cc:348] T 4b85684adaa142fba69deae1dd455697 P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.980577 12401 raft_consensus.cc:374] T 4b85684adaa142fba69deae1dd455697 P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:05.980600 12401 raft_consensus.cc:729] T 4b85684adaa142fba69deae1dd455697 P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6952e6c79a6475fb07747c4a4ace93e, State: Initialized, Role: FOLLOWER
I20260620 09:22:05.980643 12401 consensus_queue.cc:260] T 4b85684adaa142fba69deae1dd455697 P d6952e6c79a6475fb07747c4a4ace93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.980737 12401 ts_tablet_manager.cc:1428] T 4b85684adaa142fba69deae1dd455697 P d6952e6c79a6475fb07747c4a4ace93e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:05.999396 12453 raft_consensus.cc:482] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:22:05.999480 12453 raft_consensus.cc:504] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:05.999655 12453 leader_election.cc:288] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6952e6c79a6475fb07747c4a4ace93e (127.4.59.1:39897), c09e2e49b59d443c8d7b25cce0229255 (127.4.59.3:35377)
I20260620 09:22:05.999881 12047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b85684adaa142fba69deae1dd455697" candidate_uuid: "39c8f9995e714f729ff000288312d4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6952e6c79a6475fb07747c4a4ace93e" is_pre_election: true
I20260620 09:22:05.999969 12047 raft_consensus.cc:2454] T 4b85684adaa142fba69deae1dd455697 P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39c8f9995e714f729ff000288312d4f0 in term 0.
I20260620 09:22:05.999958 12311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b85684adaa142fba69deae1dd455697" candidate_uuid: "39c8f9995e714f729ff000288312d4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c09e2e49b59d443c8d7b25cce0229255" is_pre_election: true
I20260620 09:22:06.000092 12311 raft_consensus.cc:2454] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39c8f9995e714f729ff000288312d4f0 in term 0.
I20260620 09:22:06.000113 12118 leader_election.cc:302] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39c8f9995e714f729ff000288312d4f0, d6952e6c79a6475fb07747c4a4ace93e; no voters:
I20260620 09:22:06.000240 12453 raft_consensus.cc:2781] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:22:06.000298 12453 raft_consensus.cc:482] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:22:06.000319 12453 raft_consensus.cc:3037] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:06.001039 12453 raft_consensus.cc:504] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:06.001194 12453 leader_election.cc:288] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [CANDIDATE]: Term 1 election: Requested vote from peers d6952e6c79a6475fb07747c4a4ace93e (127.4.59.1:39897), c09e2e49b59d443c8d7b25cce0229255 (127.4.59.3:35377)
I20260620 09:22:06.001320 12047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b85684adaa142fba69deae1dd455697" candidate_uuid: "39c8f9995e714f729ff000288312d4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6952e6c79a6475fb07747c4a4ace93e"
I20260620 09:22:06.001394 12047 raft_consensus.cc:3037] T 4b85684adaa142fba69deae1dd455697 P d6952e6c79a6475fb07747c4a4ace93e [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:06.001372 12311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b85684adaa142fba69deae1dd455697" candidate_uuid: "39c8f9995e714f729ff000288312d4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c09e2e49b59d443c8d7b25cce0229255"
I20260620 09:22:06.001431 12311 raft_consensus.cc:3037] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:06.002163 12047 raft_consensus.cc:2454] T 4b85684adaa142fba69deae1dd455697 P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39c8f9995e714f729ff000288312d4f0 in term 1.
I20260620 09:22:06.002163 12311 raft_consensus.cc:2454] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39c8f9995e714f729ff000288312d4f0 in term 1.
I20260620 09:22:06.002329 12118 leader_election.cc:302] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39c8f9995e714f729ff000288312d4f0, c09e2e49b59d443c8d7b25cce0229255; no voters:
I20260620 09:22:06.002415 12453 raft_consensus.cc:2781] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:22:06.002481 12453 raft_consensus.cc:686] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [term 1 LEADER]: Becoming Leader. State: Replica: 39c8f9995e714f729ff000288312d4f0, State: Running, Role: LEADER
I20260620 09:22:06.002537 12453 consensus_queue.cc:237] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } }
I20260620 09:22:06.003048 11889 catalog_manager.cc:5495] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39c8f9995e714f729ff000288312d4f0 (127.4.59.2). New cstate: current_term: 1 leader_uuid: "39c8f9995e714f729ff000288312d4f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39c8f9995e714f729ff000288312d4f0" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 } health_report { overall_health: UNKNOWN } } }
I20260620 09:22:06.013911 12047 raft_consensus.cc:1261] T 4b85684adaa142fba69deae1dd455697 P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Refusing update from remote peer 39c8f9995e714f729ff000288312d4f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:22:06.013933 12311 raft_consensus.cc:1261] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Refusing update from remote peer 39c8f9995e714f729ff000288312d4f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:22:06.014118 12453 consensus_queue.cc:1035] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6952e6c79a6475fb07747c4a4ace93e" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 39897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:22:06.014201 12431 consensus_queue.cc:1035] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c09e2e49b59d443c8d7b25cce0229255" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 35377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:22:06.023581 11888 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:22:06.023661 11888 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:22:06.024144 11888 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9e18068afe5425894cf2bd7dedb13dd: Sending DeleteTablet for 3 replicas of tablet 4b85684adaa142fba69deae1dd455697
I20260620 09:22:06.024401 12167 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b85684adaa142fba69deae1dd455697 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:22:06 UTC) from {username='oryx', principal='oryx/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:39216
I20260620 09:22:06.024497 12426 tablet_replica.cc:330] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0: stopping tablet replica
I20260620 09:22:06.024503 12027 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b85684adaa142fba69deae1dd455697 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:22:06 UTC) from {username='oryx', principal='oryx/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:43212
I20260620 09:22:06.024557 12426 raft_consensus.cc:2229] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:22:06.024577 12424 tablet_replica.cc:330] T 4b85684adaa142fba69deae1dd455697 P d6952e6c79a6475fb07747c4a4ace93e: stopping tablet replica
I20260620 09:22:06.024605 12426 raft_consensus.cc:2258] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:22:06.024616 12424 raft_consensus.cc:2229] T 4b85684adaa142fba69deae1dd455697 P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:22:06.024653 12424 raft_consensus.cc:2258] T 4b85684adaa142fba69deae1dd455697 P d6952e6c79a6475fb07747c4a4ace93e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:22:06.024718 12305 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b85684adaa142fba69deae1dd455697 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:22:06 UTC) from {username='oryx', principal='oryx/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:36110
I20260620 09:22:06.024739 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 11962
I20260620 09:22:06.024803 12425 tablet_replica.cc:330] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255: stopping tablet replica
I20260620 09:22:06.024855 12425 raft_consensus.cc:2229] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:22:06.024885 12425 raft_consensus.cc:2258] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:22:06.026177 12425 ts_tablet_manager.cc:1905] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:22:06.026312 12426 ts_tablet_manager.cc:1905] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:22:06.027666 12426 ts_tablet_manager.cc:1918] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 09:22:06.027706 12426 log.cc:1196] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/4b85684adaa142fba69deae1dd455697
I20260620 09:22:06.027872 12426 ts_tablet_manager.cc:1939] T 4b85684adaa142fba69deae1dd455697 P 39c8f9995e714f729ff000288312d4f0: Deleting consensus metadata
I20260620 09:22:06.027927 12425 ts_tablet_manager.cc:1918] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 09:22:06.027979 12425 log.cc:1196] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/4b85684adaa142fba69deae1dd455697
I20260620 09:22:06.028115 11877 catalog_manager.cc:4841] TS 39c8f9995e714f729ff000288312d4f0 (127.4.59.2:40301): tablet 4b85684adaa142fba69deae1dd455697 (table test-table [id=3f88d0e5cc894b68aa1e978aeaf2218e]) successfully deleted
I20260620 09:22:06.028174 12425 ts_tablet_manager.cc:1939] T 4b85684adaa142fba69deae1dd455697 P c09e2e49b59d443c8d7b25cce0229255: Deleting consensus metadata
I20260620 09:22:06.028543 11874 catalog_manager.cc:4841] TS c09e2e49b59d443c8d7b25cce0229255 (127.4.59.3:35377): tablet 4b85684adaa142fba69deae1dd455697 (table test-table [id=3f88d0e5cc894b68aa1e978aeaf2218e]) successfully deleted
W20260620 09:22:06.031267 11874 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.59.1:39897 (error 108)
I20260620 09:22:06.031463 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 12102
I20260620 09:22:06.032099 12484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:06.031624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:39897 (local address 127.0.0.1:43226)
0620 09:22:06.032009 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:06.032053 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":49,"threads_started":1}
W20260620 09:22:06.032191 11874 catalog_manager.cc:4584] TS d6952e6c79a6475fb07747c4a4ace93e (127.4.59.1:39897): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4b85684adaa142fba69deae1dd455697: Network error: Client connection negotiation failed: client connection to 127.4.59.1:39897: connect: Connection refused (error 111)
I20260620 09:22:06.037465 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 12242
I20260620 09:22:06.044618 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 11858
2026-06-20T09:22:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2863 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:22:06 dist-test-slave-414w krb5kdc[12487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:22:06 dist-test-slave-414w krb5kdc[12487](info): set up 2 sockets
Jun 20 09:22:06 dist-test-slave-414w krb5kdc[12487](info): commencing operation
krb5kdc: starting...
W20260620 09:22:07.082101 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:22:07 dist-test-slave-414w krb5kdc[12487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:22:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:22:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:22:07.201902 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:40811
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:36885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:40811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:22:07.282172 12500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:07.282409 12500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:07.282446 12500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:07.283604 12500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:22:07.283668 12500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:07.283691 12500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:07.283720 12500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:22:07.283748 12500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:22:07.285262 12500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:36885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:40811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:40811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:07.285548 12500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:07.285770 12500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:07.290655 12506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:07.290841 12500 server_base.cc:1034] running on GCE node
W20260620 09:22:07.290655 12505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:07.290678 12508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:07.291240 12500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:07.291499 12500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:07.292641 12500 hybrid_clock.cc:648] HybridClock initialized: now 1781947327292620 us; error 42 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-20T09:22:07Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1253 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:22:07 dist-test-slave-414w krb5kdc[12513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:22:07 dist-test-slave-414w krb5kdc[12513](info): set up 2 sockets
Jun 20 09:22:07 dist-test-slave-414w krb5kdc[12513](info): commencing operation
krb5kdc: starting...
W20260620 09:22:08.345770 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:22:08 dist-test-slave-414w krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:22:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:22:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:22:08.459609 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:42745
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:42745
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:22:08.538585 12526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:08.538832 12526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:08.538872 12526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:08.540019 12526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:22:08.540087 12526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:08.540114 12526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:08.540146 12526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:22:08.540182 12526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:22:08.541642 12526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:42745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:42745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:08.541913 12526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:08.542138 12526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:08.547062 12532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:08.547049 12531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:08.547050 12534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:08.547390 12526 server_base.cc:1034] running on GCE node
I20260620 09:22:08.547643 12526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:08.547947 12526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:08.549096 12526 hybrid_clock.cc:648] HybridClock initialized: now 1781947328549079 us; error 27 us; skew 500 ppm
Jun 20 09:22:08 dist-test-slave-414w krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947328, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:22:08.559619 12526 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:22:08.560138 12526 webserver.cc:460] Webserver started at http://127.4.59.62:33269/ using document root <none> and password file <none>
I20260620 09:22:08.560375 12526 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:08.560441 12526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:08.560606 12526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:08.561759 12526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "e34385a8d67e41c4a674b007cb71010c"
format_stamp: "Formatted at 2026-06-20 09:22:08 on dist-test-slave-414w"
server_key: "f8bb7b9875c46c607b12ec675e038c23"
server_key_iv: "edff513227c47b6dff2a8aa30b2ec96b"
server_key_version: "encryptionkey@0"
I20260620 09:22:08.562119 12526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "e34385a8d67e41c4a674b007cb71010c"
format_stamp: "Formatted at 2026-06-20 09:22:08 on dist-test-slave-414w"
server_key: "f8bb7b9875c46c607b12ec675e038c23"
server_key_iv: "edff513227c47b6dff2a8aa30b2ec96b"
server_key_version: "encryptionkey@0"
I20260620 09:22:08.563462 12526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260620 09:22:08.564278 12541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:08.564713 12526 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:22:08.564817 12526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "e34385a8d67e41c4a674b007cb71010c"
format_stamp: "Formatted at 2026-06-20 09:22:08 on dist-test-slave-414w"
server_key: "f8bb7b9875c46c607b12ec675e038c23"
server_key_iv: "edff513227c47b6dff2a8aa30b2ec96b"
server_key_version: "encryptionkey@0"
I20260620 09:22:08.564889 12526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:08.573911 12526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:08.575795 12526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:08.575946 12526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:08.580083 12526 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:42745
I20260620 09:22:08.580128 12593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:42745 every 8 connection(s)
I20260620 09:22:08.580480 12526 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:22:08.581458 12594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:08.584157 12594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c: Bootstrap starting.
I20260620 09:22:08.584196 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 12526
I20260620 09:22:08.584306 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:22:08.584493 4332 external_mini_cluster.cc:1428] Setting key d29151b25fee464a5138c64d7429a609
I20260620 09:22:08.585078 12594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:08.585431 12594 log.cc:824] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c: Log is configured to *not* fsync() on all Append() calls
I20260620 09:22:08.586146 12594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c: No bootstrap required, opened a new log
I20260620 09:22:08.587937 12594 raft_consensus.cc:348] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34385a8d67e41c4a674b007cb71010c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42745 } }
I20260620 09:22:08.588091 12594 raft_consensus.cc:374] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:08.588138 12594 raft_consensus.cc:729] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e34385a8d67e41c4a674b007cb71010c, State: Initialized, Role: FOLLOWER
I20260620 09:22:08.588270 12594 consensus_queue.cc:260] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34385a8d67e41c4a674b007cb71010c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42745 } }
I20260620 09:22:08.588348 12594 raft_consensus.cc:388] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:22:08.588404 12594 raft_consensus.cc:482] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:22:08.588454 12594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:08.588994 12594 raft_consensus.cc:504] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34385a8d67e41c4a674b007cb71010c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42745 } }
I20260620 09:22:08.589113 12594 leader_election.cc:302] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e34385a8d67e41c4a674b007cb71010c; no voters:
I20260620 09:22:08.589318 12594 leader_election.cc:288] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:22:08.589365 12599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:22:08.589530 12599 raft_consensus.cc:686] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 1 LEADER]: Becoming Leader. State: Replica: e34385a8d67e41c4a674b007cb71010c, State: Running, Role: LEADER
I20260620 09:22:08.589573 12594 sys_catalog.cc:564] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:22:08.589655 12599 consensus_queue.cc:237] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34385a8d67e41c4a674b007cb71010c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42745 } }
I20260620 09:22:08.590001 12599 sys_catalog.cc:455] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e34385a8d67e41c4a674b007cb71010c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34385a8d67e41c4a674b007cb71010c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42745 } } }
I20260620 09:22:08.590180 12599 sys_catalog.cc:458] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [sys.catalog]: This master's current role is: LEADER
I20260620 09:22:08.590377 12599 sys_catalog.cc:455] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e34385a8d67e41c4a674b007cb71010c. Latest consensus state: current_term: 1 leader_uuid: "e34385a8d67e41c4a674b007cb71010c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34385a8d67e41c4a674b007cb71010c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42745 } } }
I20260620 09:22:08.590480 12599 sys_catalog.cc:458] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [sys.catalog]: This master's current role is: LEADER
Jun 20 09:22:08 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:22:08.591449 12608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:22:08.592001 12608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:22:08.593901 12608 catalog_manager.cc:1347] Generated new cluster ID: 1bd084d8c3654997be91199241618969
I20260620 09:22:08.593954 12608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:22:08.595440 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:08.585028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34832 (local address 127.4.59.62:42745)
0620 09:22:08.585239 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:08.585247 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:08.585271 (+ 24us) server_negotiation.cc:407] Connection header received
0620 09:22:08.585756 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:08.585764 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:08.585813 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:08.585960 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:08.586652 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:08.587514 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:08.588412 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:08.588871 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:08.588991 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:08.591792 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:08.591796 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:08.591797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:08.591808 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:08.593756 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:08.594166 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:08.594169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:08.594169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:08.594290 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:08.594500 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:08.594501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:08.594501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:08.594803 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:08.594955 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:08.595053 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 09:22:08.595202 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":75,"threads_started":1}
I20260620 09:22:08.614923 12608 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:22:08.615494 12608 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:22:08.621754 12608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c: Generated new TSK 0
I20260620 09:22:08.621934 12608 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:22:08.653730 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:42745
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:22:08.733774 12620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:08.733963 12620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:08.733986 12620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:08.735162 12620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:08.735213 12620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:08.735241 12620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:22:08.736725 12620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:42745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:08.736903 12620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:08.737061 12620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:08.742926 12625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:08.742911 12626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:08.742937 12628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:08.743026 12620 server_base.cc:1034] running on GCE node
I20260620 09:22:08.743314 12620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:08.743554 12620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:08.744719 12620 hybrid_clock.cc:648] HybridClock initialized: now 1781947328744702 us; error 30 us; skew 500 ppm
Jun 20 09:22:08 dist-test-slave-414w krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947328, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:22:08.747187 12620 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:22:08.747574 12620 webserver.cc:460] Webserver started at http://127.4.59.1:34463/ using document root <none> and password file <none>
I20260620 09:22:08.747788 12620 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:08.747854 12620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:08.747980 12620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:08.748953 12620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "cd1f9514a60f4cc698779effb302c871"
format_stamp: "Formatted at 2026-06-20 09:22:08 on dist-test-slave-414w"
server_key: "f92b2bb0e30e9129d4279ebef7810d73"
server_key_iv: "80fcba356a5c1dfcab5ebab689181205"
server_key_version: "encryptionkey@0"
I20260620 09:22:08.749325 12620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "cd1f9514a60f4cc698779effb302c871"
format_stamp: "Formatted at 2026-06-20 09:22:08 on dist-test-slave-414w"
server_key: "f92b2bb0e30e9129d4279ebef7810d73"
server_key_iv: "80fcba356a5c1dfcab5ebab689181205"
server_key_version: "encryptionkey@0"
I20260620 09:22:08.750558 12620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:22:08.751252 12635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:08.751428 12620 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:22:08.751521 12620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "cd1f9514a60f4cc698779effb302c871"
format_stamp: "Formatted at 2026-06-20 09:22:08 on dist-test-slave-414w"
server_key: "f92b2bb0e30e9129d4279ebef7810d73"
server_key_iv: "80fcba356a5c1dfcab5ebab689181205"
server_key_version: "encryptionkey@0"
I20260620 09:22:08.751605 12620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:08.758908 12620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:08.760985 12620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:08.761180 12620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:08.761404 12620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:22:08.761698 12620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:08.761752 12620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:08.761790 12620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:08.761821 12620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:08.767308 12620 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:33533
I20260620 09:22:08.767362 12748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:33533 every 8 connection(s)
I20260620 09:22:08.767692 12620 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:22:08.768005 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 12620
I20260620 09:22:08.768101 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:22:08.768343 4332 external_mini_cluster.cc:1428] Setting key d301019ac924bb03fe0db494ddab2759
Jun 20 09:22:08 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947328, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
I20260620 09:22:08.777055 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:08.768396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:52845 (local address 127.4.59.62:42745)
0620 09:22:08.768452 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:08.768454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:08.768678 (+ 224us) server_negotiation.cc:407] Connection header received
0620 09:22:08.769331 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:08.769333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:08.769344 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:08.769386 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:08.770253 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:08.770760 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:08.771838 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:08.772066 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:08.772153 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:08.774257 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:08.774258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:08.774259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:08.774266 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:08.775716 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:08.776113 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:08.776114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:08.776115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:08.776177 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:08.776422 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:08.776423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:08.776424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:08.776537 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:08.776614 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:08.776878 (+ 264us) server_negotiation.cc:299] Negotiation successful
0620 09:22:08.776964 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 09:22:08.777256 12751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:08.768398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:52845)
0620 09:22:08.768552 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:08.768612 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:08.769227 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:08.769450 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:08.769454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:08.769987 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:08.770157 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:08.770161 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:08.770866 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:08.770867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:08.771652 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:08.771653 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:08.772165 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:08.772166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:08.772301 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:08.772816 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:08.772824 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:08.774154 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:08.775809 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:08.775815 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:08.775817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:08.776027 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:08.776241 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:08.776242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:08.776242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:08.776352 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:08.776604 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:08.776611 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:08.776751 (+ 140us) client_negotiation.cc:769] Sending connection context
0620 09:22:08.776857 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 09:22:08.777000 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
I20260620 09:22:08.777931 12749 heartbeater.cc:347] Connected to a master server at 127.4.59.62:42745
I20260620 09:22:08.778024 12749 heartbeater.cc:464] Registering TS with master...
I20260620 09:22:08.778204 12749 heartbeater.cc:511] Master 127.4.59.62:42745 requested a full tablet report, sending...
I20260620 09:22:08.778725 12558 ts_manager.cc:194] Registered new tserver with Master: cd1f9514a60f4cc698779effb302c871 (127.4.59.1:33533)
I20260620 09:22:08.779333 12558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:52845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:22:08.791817 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:42745
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:22:08.871335 12754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:08.871539 12754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:08.871562 12754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:08.872754 12754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:08.872797 12754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:08.872824 12754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:22:08.874284 12754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:42745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:08.874532 12754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:08.874761 12754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:08.879769 12762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:08.879835 12759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:08.879874 12754 server_base.cc:1034] running on GCE node
W20260620 09:22:08.879823 12760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:08.880198 12754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:08.880446 12754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:08.881608 12754 hybrid_clock.cc:648] HybridClock initialized: now 1781947328881592 us; error 44 us; skew 500 ppm
Jun 20 09:22:08 dist-test-slave-414w krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947328, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:22:08.884109 12754 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:22:08.884562 12754 webserver.cc:460] Webserver started at http://127.4.59.2:37075/ using document root <none> and password file <none>
I20260620 09:22:08.884804 12754 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:08.884866 12754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:08.885006 12754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:08.886190 12754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "e89814486226419f8225e1e895771038"
format_stamp: "Formatted at 2026-06-20 09:22:08 on dist-test-slave-414w"
server_key: "18cc912026040bc930d841faa0cbedf1"
server_key_iv: "85a71415366d69789ae8c7662ff03dcc"
server_key_version: "encryptionkey@0"
I20260620 09:22:08.886554 12754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "e89814486226419f8225e1e895771038"
format_stamp: "Formatted at 2026-06-20 09:22:08 on dist-test-slave-414w"
server_key: "18cc912026040bc930d841faa0cbedf1"
server_key_iv: "85a71415366d69789ae8c7662ff03dcc"
server_key_version: "encryptionkey@0"
I20260620 09:22:08.887842 12754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:22:08.888547 12769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:08.888720 12754 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:22:08.888813 12754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "e89814486226419f8225e1e895771038"
format_stamp: "Formatted at 2026-06-20 09:22:08 on dist-test-slave-414w"
server_key: "18cc912026040bc930d841faa0cbedf1"
server_key_iv: "85a71415366d69789ae8c7662ff03dcc"
server_key_version: "encryptionkey@0"
I20260620 09:22:08.888893 12754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:08.912422 12754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:08.914431 12754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:08.914593 12754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:08.914837 12754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:22:08.915155 12754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:08.915208 12754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:08.915249 12754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:08.915277 12754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:08.921249 12754 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:42413
I20260620 09:22:08.921296 12882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:42413 every 8 connection(s)
I20260620 09:22:08.921597 12754 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:22:08.926290 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 12754
I20260620 09:22:08.926430 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:22:08.926656 4332 external_mini_cluster.cc:1428] Setting key 32e6bb0a0c2e21e31af26bd08ae1c7db
Jun 20 09:22:08 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947328, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:22:08.931248 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:08.922320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:55151 (local address 127.4.59.62:42745)
0620 09:22:08.922380 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:08.922383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:08.922563 (+ 180us) server_negotiation.cc:407] Connection header received
0620 09:22:08.923091 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:08.923092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:08.923104 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:08.923159 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:08.924080 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:08.924588 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:08.925671 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:08.925872 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:08.925958 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:08.928363 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:08.928364 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:08.928364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:08.928371 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:08.929904 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:08.930299 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:08.930300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:08.930301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:08.930362 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:08.930608 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:08.930609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:08.930609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:08.930726 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:08.930803 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:08.931067 (+ 264us) server_negotiation.cc:299] Negotiation successful
0620 09:22:08.931146 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 09:22:08.931447 12885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:08.922247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:55151)
0620 09:22:08.922435 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:08.922480 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:08.922986 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:08.923226 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:08.923230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:08.923797 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:08.923979 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:08.923983 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:08.924682 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:08.924683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:08.925572 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:08.925574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:08.925968 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:08.925969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:08.926135 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:08.926737 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:08.926749 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:08.928258 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:08.929993 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:08.929999 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:08.930001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:08.930215 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:08.930427 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:08.930428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:08.930429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:08.930540 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:08.930794 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:08.930800 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:08.930937 (+ 137us) client_negotiation.cc:769] Sending connection context
0620 09:22:08.931045 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 09:22:08.931198 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260620 09:22:08.931933 12883 heartbeater.cc:347] Connected to a master server at 127.4.59.62:42745
I20260620 09:22:08.932013 12883 heartbeater.cc:464] Registering TS with master...
I20260620 09:22:08.932219 12883 heartbeater.cc:511] Master 127.4.59.62:42745 requested a full tablet report, sending...
I20260620 09:22:08.932600 12558 ts_manager.cc:194] Registered new tserver with Master: e89814486226419f8225e1e895771038 (127.4.59.2:42413)
I20260620 09:22:08.933133 12558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:55151
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:22:08.952013 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:42745
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:22:09.031876 12888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:09.032063 12888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:09.032084 12888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:09.033221 12888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:09.033259 12888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:09.033285 12888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:22:09.034627 12888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:42745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:09.034796 12888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:09.034957 12888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:09.039954 12896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:09.039963 12893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:09.040112 12888 server_base.cc:1034] running on GCE node
W20260620 09:22:09.039963 12894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:09.040417 12888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:09.040670 12888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:09.041857 12888 hybrid_clock.cc:648] HybridClock initialized: now 1781947329041840 us; error 26 us; skew 500 ppm
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947329, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:22:09.044343 12888 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:22:09.044703 12888 webserver.cc:460] Webserver started at http://127.4.59.3:37237/ using document root <none> and password file <none>
I20260620 09:22:09.044880 12888 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:09.044922 12888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:09.045008 12888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:09.046072 12888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "b5d4a073f9d64cd9a90fab57357ea101"
format_stamp: "Formatted at 2026-06-20 09:22:09 on dist-test-slave-414w"
server_key: "e8a1851d380bbd20308caf981caa0780"
server_key_iv: "76e75be108b4e40b080c3c55adb687f2"
server_key_version: "encryptionkey@0"
I20260620 09:22:09.046447 12888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "b5d4a073f9d64cd9a90fab57357ea101"
format_stamp: "Formatted at 2026-06-20 09:22:09 on dist-test-slave-414w"
server_key: "e8a1851d380bbd20308caf981caa0780"
server_key_iv: "76e75be108b4e40b080c3c55adb687f2"
server_key_version: "encryptionkey@0"
I20260620 09:22:09.047753 12888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:22:09.048450 12903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:09.048615 12888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:22:09.048695 12888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "b5d4a073f9d64cd9a90fab57357ea101"
format_stamp: "Formatted at 2026-06-20 09:22:09 on dist-test-slave-414w"
server_key: "e8a1851d380bbd20308caf981caa0780"
server_key_iv: "76e75be108b4e40b080c3c55adb687f2"
server_key_version: "encryptionkey@0"
I20260620 09:22:09.048760 12888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:09.061395 12888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:09.063537 12888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:09.063645 12888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:09.063820 12888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:22:09.064054 12888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:09.064083 12888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:09.064105 12888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:09.064116 12888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:09.069794 12888 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:43419
I20260620 09:22:09.069855 13016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:43419 every 8 connection(s)
I20260620 09:22:09.070153 12888 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947329, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:22:09.076828 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 12888
I20260620 09:22:09.076936 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:22:09.077204 4332 external_mini_cluster.cc:1428] Setting key c28baf371221970a1aa685b236802daa
I20260620 09:22:09.079591 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:09.070867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:38495 (local address 127.4.59.62:42745)
0620 09:22:09.070944 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:09.070946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:09.071134 (+ 188us) server_negotiation.cc:407] Connection header received
0620 09:22:09.071668 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:09.071668 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:09.071680 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:09.071743 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:09.072659 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.073221 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.074204 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.074397 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.074484 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.076453 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:09.076454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:09.076454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:09.076460 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:09.078453 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:09.078814 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:09.078815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:09.078816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:09.078867 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:09.079084 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:09.079084 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:09.079084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:09.079185 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:09.079239 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:09.079458 (+ 219us) server_negotiation.cc:299] Negotiation successful
0620 09:22:09.079512 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 09:22:09.079761 13019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:09.070792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:38495)
0620 09:22:09.070999 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:09.071047 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:09.071560 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:09.071809 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:09.071813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:09.072376 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:09.072555 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.072560 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.073318 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.073319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.074100 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.074102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.074504 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.074505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.074625 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.075045 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:09.075053 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:09.076373 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:09.078526 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:09.078531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:09.078533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:09.078748 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:09.078930 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:09.078931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:09.078931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:09.079025 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:09.079245 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:09.079249 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:09.079354 (+ 105us) client_negotiation.cc:769] Sending connection context
0620 09:22:09.079433 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 09:22:09.079543 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":84,"threads_started":1}
I20260620 09:22:09.080166 13017 heartbeater.cc:347] Connected to a master server at 127.4.59.62:42745
I20260620 09:22:09.080271 13017 heartbeater.cc:464] Registering TS with master...
I20260620 09:22:09.080443 13017 heartbeater.cc:511] Master 127.4.59.62:42745 requested a full tablet report, sending...
I20260620 09:22:09.080930 12558 ts_manager.cc:194] Registered new tserver with Master: b5d4a073f9d64cd9a90fab57357ea101 (127.4.59.3:43419)
I20260620 09:22:09.081431 12558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:38495
I20260620 09:22:09.088922 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.1.keytab.
Entry for principal oryx/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.1.keytab.
WARNING: no policy specified for oryx/127.4.59.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.2.keytab.
Entry for principal oryx/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.2.keytab.
WARNING: no policy specified for oryx/127.4.59.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.3.keytab.
Entry for principal oryx/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.3.keytab.
I20260620 09:22:09.159997 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 12620
I20260620 09:22:09.164848 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 12754
I20260620 09:22:09.169225 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 12888
I20260620 09:22:09.173377 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 12526
I20260620 09:22:09.178508 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:42745
--webserver_interface=127.4.59.62
--webserver_port=33269
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:42745
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:22:09.257853 13026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:09.258074 13026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:09.258109 13026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:09.259264 13026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:22:09.259331 13026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:09.259356 13026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:09.259384 13026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:22:09.259405 13026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:22:09.260885 13026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:42745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:42745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=33269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:09.261174 13026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:09.261396 13026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:09.266234 13032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:09.266265 13026 server_base.cc:1034] running on GCE node
W20260620 09:22:09.266227 13034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:09.266237 13031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:09.266774 13026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:09.267071 13026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:09.268240 13026 hybrid_clock.cc:648] HybridClock initialized: now 1781947329268220 us; error 52 us; skew 500 ppm
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947329, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:22:09.270803 13026 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:22:09.271309 13026 webserver.cc:460] Webserver started at http://127.4.59.62:33269/ using document root <none> and password file <none>
I20260620 09:22:09.271543 13026 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:09.271624 13026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:09.273017 13026 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:22:09.273711 13041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:09.273887 13026 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:09.273986 13026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "e34385a8d67e41c4a674b007cb71010c"
format_stamp: "Formatted at 2026-06-20 09:22:08 on dist-test-slave-414w"
server_key: "f8bb7b9875c46c607b12ec675e038c23"
server_key_iv: "edff513227c47b6dff2a8aa30b2ec96b"
server_key_version: "encryptionkey@0"
I20260620 09:22:09.274349 13026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:09.287647 13026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:09.290062 13026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:09.290227 13026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:09.294653 13026 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:42745
I20260620 09:22:09.294687 13093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:42745 every 8 connection(s)
I20260620 09:22:09.295063 13026 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:22:09.296535 13094 sys_catalog.cc:263] Verifying existing consensus state
I20260620 09:22:09.297211 13094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c: Bootstrap starting.
I20260620 09:22:09.302986 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 13026
I20260620 09:22:09.303359 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:33533
--local_ip_for_outbound_sockets=127.4.59.1
--tserver_master_addrs=127.4.59.62:42745
--webserver_port=34463
--webserver_interface=127.4.59.1
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.59.1
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.1.keytab with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:22:09.305833 13094 log.cc:824] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c: Log is configured to *not* fsync() on all Append() calls
I20260620 09:22:09.307538 13094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 09:22:09.307828 13094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c: Bootstrap complete.
I20260620 09:22:09.310092 13094 raft_consensus.cc:348] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34385a8d67e41c4a674b007cb71010c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42745 } }
I20260620 09:22:09.310305 13094 raft_consensus.cc:729] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e34385a8d67e41c4a674b007cb71010c, State: Initialized, Role: FOLLOWER
I20260620 09:22:09.310470 13094 consensus_queue.cc:260] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34385a8d67e41c4a674b007cb71010c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42745 } }
I20260620 09:22:09.310541 13094 raft_consensus.cc:388] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:22:09.310603 13094 raft_consensus.cc:482] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:22:09.310649 13094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 1 FOLLOWER]: Advancing to term 2
I20260620 09:22:09.311609 13094 raft_consensus.cc:504] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34385a8d67e41c4a674b007cb71010c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42745 } }
I20260620 09:22:09.311693 13094 leader_election.cc:302] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e34385a8d67e41c4a674b007cb71010c; no voters:
I20260620 09:22:09.311889 13094 leader_election.cc:288] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 09:22:09.311960 13098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 2 FOLLOWER]: Leader election won for term 2
I20260620 09:22:09.312134 13098 raft_consensus.cc:686] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [term 2 LEADER]: Becoming Leader. State: Replica: e34385a8d67e41c4a674b007cb71010c, State: Running, Role: LEADER
I20260620 09:22:09.312161 13094 sys_catalog.cc:564] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:22:09.312254 13098 consensus_queue.cc:237] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34385a8d67e41c4a674b007cb71010c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42745 } }
I20260620 09:22:09.312551 13098 sys_catalog.cc:455] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e34385a8d67e41c4a674b007cb71010c. Latest consensus state: current_term: 2 leader_uuid: "e34385a8d67e41c4a674b007cb71010c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34385a8d67e41c4a674b007cb71010c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42745 } } }
I20260620 09:22:09.312552 13099 sys_catalog.cc:455] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e34385a8d67e41c4a674b007cb71010c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34385a8d67e41c4a674b007cb71010c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42745 } } }
I20260620 09:22:09.312642 13098 sys_catalog.cc:458] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [sys.catalog]: This master's current role is: LEADER
I20260620 09:22:09.312670 13099 sys_catalog.cc:458] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c [sys.catalog]: This master's current role is: LEADER
I20260620 09:22:09.312975 13105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:22:09.313563 13105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:22:09.313709 13105 catalog_manager.cc:1259] Loaded cluster ID: 1bd084d8c3654997be91199241618969
I20260620 09:22:09.313747 13105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:22:09.315110 13105 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:22:09.315287 13105 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e34385a8d67e41c4a674b007cb71010c: Loaded TSK: 0
I20260620 09:22:09.315493 13105 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 09:22:09.386622 13096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:09.386811 13096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:09.386834 13096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:09.387990 13096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:09.388033 13096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:09.388058 13096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:22:09.389542 13096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.1.keytab
--principal=oryx/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:33533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=34463
--tserver_master_addrs=127.4.59.62:42745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:09.389770 13096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:09.390009 13096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:09.394676 13120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:09.394707 13121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:09.394850 13123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:09.394963 13096 server_base.cc:1034] running on GCE node
I20260620 09:22:09.395149 13096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:09.395421 13096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:09.396565 13096 hybrid_clock.cc:648] HybridClock initialized: now 1781947329396550 us; error 31 us; skew 500 ppm
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947329, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:22:09.399165 13096 init.cc:377] Logged in from keytab as oryx/127.4.59.1@KRBTEST.COM (short username oryx)
I20260620 09:22:09.399639 13096 webserver.cc:460] Webserver started at http://127.4.59.1:34463/ using document root <none> and password file <none>
I20260620 09:22:09.399892 13096 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:09.399974 13096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:09.401330 13096 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:22:09.401939 13130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:09.402128 13096 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 09:22:09.402226 13096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "cd1f9514a60f4cc698779effb302c871"
format_stamp: "Formatted at 2026-06-20 09:22:08 on dist-test-slave-414w"
server_key: "f92b2bb0e30e9129d4279ebef7810d73"
server_key_iv: "80fcba356a5c1dfcab5ebab689181205"
server_key_version: "encryptionkey@0"
I20260620 09:22:09.402556 13096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:09.415786 13096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:09.418037 13096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:09.418196 13096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:09.418416 13096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:22:09.418777 13096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:09.418835 13096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:09.418874 13096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:09.418901 13096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:09.424758 13096 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:33533
I20260620 09:22:09.424813 13243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:33533 every 8 connection(s)
I20260620 09:22:09.425130 13096 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:22:09.428797 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 13096
I20260620 09:22:09.429092 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:42413
--local_ip_for_outbound_sockets=127.4.59.2
--tserver_master_addrs=127.4.59.62:42745
--webserver_port=37075
--webserver_interface=127.4.59.2
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.59.2
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.2.keytab with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:09.434454 13246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:09.425881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:37757)
0620 09:22:09.426095 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:09.426140 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:09.426733 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:09.427319 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:09.427322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:09.427865 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:09.428043 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.428048 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.429668 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.429669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.430789 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.430792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.431426 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.431427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.431552 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.432063 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:09.432071 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:09.434236 (+ 2165us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":115,"threads_started":1}
W20260620 09:22:09.434549 13247 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:09.426010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:37757 (local address 127.4.59.62:42745)
0620 09:22:09.426925 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:09.426933 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:09.426966 (+ 33us) server_negotiation.cc:407] Connection header received
0620 09:22:09.427076 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:09.427082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:09.427130 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:09.427254 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:09.428166 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.429565 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.430915 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.431322 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.431450 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.434364 (+ 2914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:37757: BlockingRecv error: recv got EOF from 127.4.59.1:37757 (error 108)
Metrics: {"server-negotiator.queue_time_us":868,"thread_start_us":70,"threads_started":1}
W20260620 09:22:09.434940 13244 heartbeater.cc:644] Failed to heartbeat to 127.4.59.62:42745 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.59.62:42745: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:09.439208 13246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:09.435164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:42965)
0620 09:22:09.435249 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:09.435260 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:09.435304 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:09.435454 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:09.435454 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:09.435580 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:09.435621 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.435622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.436229 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.436231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.436727 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.436728 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.437083 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.437084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.437177 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.437581 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:09.437585 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:09.439128 (+ 1543us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260620 09:22:09.439249 13247 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:09.435213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:42965 (local address 127.4.59.62:42745)
0620 09:22:09.435247 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:09.435249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:09.435290 (+ 41us) server_negotiation.cc:407] Connection header received
0620 09:22:09.435360 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:09.435361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:09.435372 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:09.435451 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:09.435710 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.436153 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.436808 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.436997 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.437080 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.439194 (+ 2114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:42965: BlockingRecv error: recv got EOF from 127.4.59.1:42965 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:09.443487 13246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:09.439685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:56323)
0620 09:22:09.439722 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:09.439732 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:09.439763 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:09.439878 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:09.439879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:09.440002 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:09.440024 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.440025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.440609 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.440610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.441084 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.441085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.441463 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.441464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.441514 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.441911 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:09.441916 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:09.443415 (+ 1499us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":24}
W20260620 09:22:09.443655 13244 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 09:22:09.443535 13247 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:09.439745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:56323 (local address 127.4.59.62:42745)
0620 09:22:09.439784 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:09.439787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:09.439807 (+ 20us) server_negotiation.cc:407] Connection header received
0620 09:22:09.439825 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:09.439826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:09.439834 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:09.439877 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:09.440095 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.440531 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.441179 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.441380 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.441461 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.443476 (+ 2015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:56323: BlockingRecv error: recv got EOF from 127.4.59.1:56323 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260620 09:22:09.516016 13248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:09.516224 13248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:09.516247 13248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:09.517421 13248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:09.517462 13248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:09.517488 13248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:22:09.518918 13248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.2.keytab
--principal=oryx/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:42413
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=37075
--tserver_master_addrs=127.4.59.62:42745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:09.519165 13248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:09.519409 13248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:09.524431 13256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:09.524447 13254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:09.524435 13253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:09.524487 13248 server_base.cc:1034] running on GCE node
I20260620 09:22:09.524806 13248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:09.524994 13248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:09.526117 13248 hybrid_clock.cc:648] HybridClock initialized: now 1781947329526102 us; error 35 us; skew 500 ppm
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947329, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:22:09.528506 13248 init.cc:377] Logged in from keytab as oryx/127.4.59.2@KRBTEST.COM (short username oryx)
I20260620 09:22:09.528939 13248 webserver.cc:460] Webserver started at http://127.4.59.2:37075/ using document root <none> and password file <none>
I20260620 09:22:09.529214 13248 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:09.529295 13248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:09.530593 13248 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:22:09.531219 13263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:09.531435 13248 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:22:09.531534 13248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "e89814486226419f8225e1e895771038"
format_stamp: "Formatted at 2026-06-20 09:22:08 on dist-test-slave-414w"
server_key: "18cc912026040bc930d841faa0cbedf1"
server_key_iv: "85a71415366d69789ae8c7662ff03dcc"
server_key_version: "encryptionkey@0"
I20260620 09:22:09.531870 13248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:09.538713 13248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:09.540848 13248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:09.541002 13248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:09.541256 13248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:22:09.541553 13248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:09.541608 13248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:09.541644 13248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:09.541671 13248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:09.547215 13248 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:42413
I20260620 09:22:09.547282 13376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:42413 every 8 connection(s)
I20260620 09:22:09.547612 13248 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:09.554454 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 13248
I20260620 09:22:09.554788 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:43419
--local_ip_for_outbound_sockets=127.4.59.3
--tserver_master_addrs=127.4.59.62:42745
--webserver_port=37237
--webserver_interface=127.4.59.3
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.59.3
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.3.keytab with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:22:09.555079 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:09.548332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:45111)
0620 09:22:09.548491 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:09.548524 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:09.549002 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:09.549306 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:09.549310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:09.549853 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:09.550028 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.550033 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.550722 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.550723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.551519 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.551521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.551919 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.551920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.552049 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.552491 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:09.552499 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:09.554864 (+ 2365us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
W20260620 09:22:09.555173 13247 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:09.548343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:45111 (local address 127.4.59.62:42745)
0620 09:22:09.548387 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:09.548389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:09.548626 (+ 237us) server_negotiation.cc:407] Connection header received
0620 09:22:09.549104 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:09.549105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:09.549121 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:09.549231 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:09.550144 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.550629 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.551620 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.551806 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.551908 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.555095 (+ 3187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:45111: BlockingRecv error: recv got EOF from 127.4.59.2:45111 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 09:22:09.556342 13377 heartbeater.cc:644] Failed to heartbeat to 127.4.59.62:42745 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.59.62:42745: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
W20260620 09:22:09.562609 13247 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:09.556681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:54813 (local address 127.4.59.62:42745)
0620 09:22:09.556827 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:09.556829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:09.556838 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:22:09.556861 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:09.556862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:09.556889 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:09.556975 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:09.557448 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.558183 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.559244 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.559604 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.559675 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.562527 (+ 2852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:54813: BlockingRecv error: recv got EOF from 127.4.59.2:54813 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
I20260620 09:22:09.562795 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:09.556591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:54813)
0620 09:22:09.556634 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:09.556644 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:09.556681 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:09.556990 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:09.556991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:09.557247 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:09.557304 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.557306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.558291 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.558292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.559127 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.559129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.559732 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.559733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.559814 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.560358 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:09.560364 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:09.562708 (+ 2344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:09.568547 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:09.563574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:43641)
0620 09:22:09.563980 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:09.563993 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:09.564050 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:09.564241 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:09.564242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:09.564392 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:09.564439 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.564441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.565336 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.565337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.565887 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.565889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.566265 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.566266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.566318 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.566745 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:09.566750 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:09.568502 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386}
W20260620 09:22:09.568670 13247 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:09.563631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:43641 (local address 127.4.59.62:42745)
0620 09:22:09.563665 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:09.563667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:09.564062 (+ 395us) server_negotiation.cc:407] Connection header received
0620 09:22:09.564115 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:09.564132 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:09.564145 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:09.564217 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:09.564526 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.565244 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.565977 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.566183 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.566264 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.568609 (+ 2345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:43641: BlockingRecv error: recv got EOF from 127.4.59.2:43641 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260620 09:22:09.568924 13377 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 09:22:09.638264 13380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:09.638463 13380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:09.638486 13380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:09.639569 13380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:09.639602 13380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:09.639629 13380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:22:09.640950 13380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/krb5kdc/oryx_127.4.59.3.keytab
--principal=oryx/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:43419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=37237
--tserver_master_addrs=127.4.59.62:42745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:09.641125 13380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:09.641377 13380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:09.646281 13388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:09.646291 13385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:09.646404 13386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:09.646636 13380 server_base.cc:1034] running on GCE node
I20260620 09:22:09.646865 13380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:09.647112 13380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:09.648293 13380 hybrid_clock.cc:648] HybridClock initialized: now 1781947329648272 us; error 29 us; skew 500 ppm
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947329, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:22:09.651120 13380 init.cc:377] Logged in from keytab as oryx/127.4.59.3@KRBTEST.COM (short username oryx)
I20260620 09:22:09.651563 13380 webserver.cc:460] Webserver started at http://127.4.59.3:37237/ using document root <none> and password file <none>
I20260620 09:22:09.651809 13380 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:09.651894 13380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:09.653295 13380 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:22:09.653973 13395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:09.654199 13380 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 09:22:09.654330 13380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "b5d4a073f9d64cd9a90fab57357ea101"
format_stamp: "Formatted at 2026-06-20 09:22:09 on dist-test-slave-414w"
server_key: "e8a1851d380bbd20308caf981caa0780"
server_key_iv: "76e75be108b4e40b080c3c55adb687f2"
server_key_version: "encryptionkey@0"
I20260620 09:22:09.654717 13380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:09.662210 13380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:09.664450 13380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:09.664610 13380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:09.664844 13380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:22:09.665175 13380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:09.665237 13380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:09.665277 13380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:09.665304 13380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:09.671845 13380 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:43419
I20260620 09:22:09.671967 13508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:43419 every 8 connection(s)
I20260620 09:22:09.672272 13380 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
W20260620 09:22:09.679625 13247 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:09.673032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:43379 (local address 127.4.59.62:42745)
0620 09:22:09.673079 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:09.673081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:09.673344 (+ 263us) server_negotiation.cc:407] Connection header received
0620 09:22:09.673880 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:09.673881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:09.673893 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:09.673956 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:09.674904 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.675391 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.676366 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.676556 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.676641 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.679565 (+ 2924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:43379: BlockingRecv error: recv got EOF from 127.4.59.3:43379 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 09:22:09.679694 13511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:09.673017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:43379)
0620 09:22:09.673212 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:09.673258 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:09.673752 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:09.674034 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:09.674038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:09.674603 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:09.674798 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.674802 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.675490 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.675492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.676275 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.676277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.676651 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.676653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.676778 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.677408 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:09.677417 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:09.679499 (+ 2082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":82,"threads_started":1}
W20260620 09:22:09.680047 13509 heartbeater.cc:644] Failed to heartbeat to 127.4.59.62:42745 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.59.62:42745: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
I20260620 09:22:09.680312 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 13380
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:09.686136 13511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:09.680388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:32997)
0620 09:22:09.680442 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:09.680453 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:09.680492 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:09.680958 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:09.680959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:09.681113 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:09.681190 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.681192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.682220 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.682221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.682997 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.682999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.683432 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.683433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.683481 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.683940 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:09.683944 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:09.686071 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 09:22:09.686210 13247 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:09.680542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:32997 (local address 127.4.59.62:42745)
0620 09:22:09.680579 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:09.680581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:09.680589 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:22:09.680870 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:09.680871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:09.680883 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:09.680943 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:09.681395 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.682135 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.683080 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.683357 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.683428 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.686138 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:32997: BlockingRecv error: recv got EOF from 127.4.59.3:32997 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 09:22:09.689863 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:09.680684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41476 (local address 127.4.59.62:42745)
0620 09:22:09.680814 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:09.680816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:09.680940 (+ 124us) server_negotiation.cc:407] Connection header received
0620 09:22:09.680989 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:09.680990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:09.681002 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:09.681070 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:09.682035 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.682769 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.684140 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.684476 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.684537 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.686104 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:09.686108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:09.686109 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:09.686122 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:09.688263 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:09.688660 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:09.688663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:09.688663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:09.688794 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:09.689017 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:09.689018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:09.689019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:09.689382 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:09.689532 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:09.689640 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 09:22:09.689740 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":52,"threads_started":1}
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:09 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
W20260620 09:22:09.691123 13247 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:09.686514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:43559 (local address 127.4.59.62:42745)
0620 09:22:09.686557 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:09.686560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:09.686568 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:22:09.686649 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:09.686649 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:09.686656 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:09.686699 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:09.687070 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.687516 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.688214 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.688506 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.688580 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.691085 (+ 2505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:43559: BlockingRecv error: recv got EOF from 127.4.59.3:43559 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 09:22:09.691304 13511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:09.686452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:43559)
0620 09:22:09.686518 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:09.686529 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:09.686578 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:09.686705 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:09.686706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:09.686921 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:09.686949 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.686951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.687625 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.687626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.688115 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:09.688117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:09.688589 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:09.688590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:09.688661 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:09.689103 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:09.689107 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:09.691231 (+ 2124us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260620 09:22:09.691454 13509 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 20 09:22:10 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:10 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:10.448551 13514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:10.443984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:50341)
0620 09:22:10.444110 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:10.444121 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:10.444175 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:10.444332 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:10.444333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:10.444481 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:10.444527 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:10.444529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:10.445203 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:10.445204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:10.445812 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:10.445814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:10.446136 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:10.446137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:10.446179 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:10.446657 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:10.446662 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:10.448491 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
W20260620 09:22:10.448645 13515 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:10.444071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:50341 (local address 127.4.59.62:42745)
0620 09:22:10.444193 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:10.444195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:10.444204 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:22:10.444223 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:10.444224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:10.444233 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:10.444295 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:10.444602 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:10.445084 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:10.445882 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:10.446070 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:10.446122 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:10.448574 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:50341: BlockingRecv error: recv got EOF from 127.4.59.1:50341 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":71,"threads_started":1}
Jun 20 09:22:10 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:10 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:10.573781 13516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:10.569323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:54443)
0620 09:22:10.569474 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:10.569487 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:10.569567 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:10.569719 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:10.569720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:10.569885 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:10.569937 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:10.569939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:10.570588 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:10.570589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:10.571122 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:10.571124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:10.571513 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:10.571514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:10.571577 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:10.572007 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:10.572011 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:10.573716 (+ 1705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
W20260620 09:22:10.573899 13515 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:10.569396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:54443 (local address 127.4.59.62:42745)
0620 09:22:10.569467 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:10.569469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:10.569549 (+ 80us) server_negotiation.cc:407] Connection header received
0620 09:22:10.569648 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:10.569649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:10.569663 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:10.569716 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:10.570013 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:10.570506 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:10.571217 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:10.571425 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:10.571512 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:10.573802 (+ 2290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:54443: BlockingRecv error: recv got EOF from 127.4.59.2:54443 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 20 09:22:10 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:10 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:10.696561 13517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:10.691841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:54657)
0620 09:22:10.692014 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:10.692030 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:10.692122 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:10.692280 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:10.692280 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:10.692458 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:10.692503 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:10.692505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:10.693188 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:10.693190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:10.693728 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:10.693729 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:10.694117 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:10.694118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:10.694182 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:10.694630 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:10.694635 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:10.696455 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":101,"threads_started":1}
W20260620 09:22:10.696596 13515 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:10.691958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:54657 (local address 127.4.59.62:42745)
0620 09:22:10.692041 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:10.692043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:10.692102 (+ 59us) server_negotiation.cc:407] Connection header received
0620 09:22:10.692197 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:10.692198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:10.692213 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:10.692260 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:10.692583 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:10.693081 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:10.693824 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:10.694030 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:10.694115 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:10.696531 (+ 2416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:54657: BlockingRecv error: recv got EOF from 127.4.59.3:54657 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 20 09:22:11 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:11 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:11.453881 13518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:11.449039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:41563)
0620 09:22:11.449215 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:11.449229 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:11.449289 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:11.449573 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:11.449573 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:11.449731 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:11.449778 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:11.449780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:11.450461 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:11.450461 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:11.450983 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:11.450985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:11.451316 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:11.451317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:11.451358 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:11.451820 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:11.451824 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:11.453781 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":76,"threads_started":1}
W20260620 09:22:11.453946 13519 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:11.449235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:41563 (local address 127.4.59.62:42745)
0620 09:22:11.449454 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:11.449456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:11.449464 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:22:11.449485 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:11.449485 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:11.449495 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:11.449553 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:11.449866 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:11.450389 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:11.451066 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:11.451255 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:11.451302 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:11.453869 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:41563: BlockingRecv error: recv got EOF from 127.4.59.1:41563 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":160,"threads_started":1}
Jun 20 09:22:11 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:11 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:11.579497 13520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:11.574281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:56743)
0620 09:22:11.574422 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:11.574433 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:11.574489 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:11.574632 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:11.574633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:11.574790 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:11.574839 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:11.574840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:11.575681 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:11.575682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:11.576233 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:11.576234 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:11.576731 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:11.576736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:11.576778 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:11.577284 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:11.577289 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:11.579415 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
W20260620 09:22:11.579612 13519 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:11.574369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:56743 (local address 127.4.59.62:42745)
0620 09:22:11.574420 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:11.574422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:11.574473 (+ 51us) server_negotiation.cc:407] Connection header received
0620 09:22:11.574534 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:11.574535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:11.574545 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:11.574611 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:11.574905 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:11.575598 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:11.576383 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:11.576656 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:11.576718 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:11.579514 (+ 2796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:56743: BlockingRecv error: recv got EOF from 127.4.59.2:56743 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Jun 20 09:22:11 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:11 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:11.701499 13521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:11.697086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:60941)
0620 09:22:11.697245 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:11.697256 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:11.697313 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:11.697449 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:11.697449 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:11.697612 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:11.697660 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:11.697662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:11.698305 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:11.698305 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:11.698835 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:11.698837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:11.699172 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:11.699173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:11.699223 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:11.699665 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:11.699669 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:11.701419 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
W20260620 09:22:11.701550 13519 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:11.697223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:60941 (local address 127.4.59.62:42745)
0620 09:22:11.697289 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:11.697291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:11.697303 (+ 12us) server_negotiation.cc:407] Connection header received
0620 09:22:11.697357 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:11.697358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:11.697369 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:11.697433 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:11.697748 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:11.698236 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:11.698907 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:11.699088 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:11.699142 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:11.701502 (+ 2360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:60941: BlockingRecv error: recv got EOF from 127.4.59.3:60941 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 20 09:22:12 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:12 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:12.459535 13522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:12.454373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:33407)
0620 09:22:12.454548 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:12.454560 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:12.454622 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:12.454903 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:12.454904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:12.455069 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:12.455118 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:12.455120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:12.455768 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:12.455769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:12.456314 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:12.456315 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:12.456682 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:12.456683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:12.456733 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:12.457279 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:12.457283 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:12.459427 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":98,"threads_started":1}
W20260620 09:22:12.459584 13523 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:12.454477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:33407 (local address 127.4.59.62:42745)
0620 09:22:12.454687 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:12.454689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:12.454700 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:22:12.454727 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:12.454727 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:12.454740 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:12.454815 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:12.455195 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:12.455697 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:12.456396 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:12.456595 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:12.456654 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:12.459508 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:33407: BlockingRecv error: recv got EOF from 127.4.59.1:33407 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":209,"threads_started":1}
Jun 20 09:22:12 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:12 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:12.584990 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:12.580051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:43159)
0620 09:22:12.580278 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:12.580291 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:12.580387 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:12.580515 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:12.580516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:12.580689 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:12.580744 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:12.580746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:12.581607 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:12.581608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:12.582168 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:12.582169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:12.582539 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:12.582540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:12.582600 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:12.583093 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:12.583098 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:12.584930 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":86,"threads_started":1}
W20260620 09:22:12.585060 13523 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:12.580091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:43159 (local address 127.4.59.62:42745)
0620 09:22:12.580185 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:12.580187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:12.580369 (+ 182us) server_negotiation.cc:407] Connection header received
0620 09:22:12.580447 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:12.580448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:12.580461 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:12.580512 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:12.580842 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:12.581483 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:12.582273 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:12.582465 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:12.582542 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:12.585013 (+ 2471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:43159: BlockingRecv error: recv got EOF from 127.4.59.2:43159 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 20 09:22:12 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:12 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:12.706461 13525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:12.702123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:52267)
0620 09:22:12.702296 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:12.702307 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:12.702370 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:12.702481 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:12.702482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:12.702649 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:12.702700 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:12.702702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:12.703324 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:12.703326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:12.703847 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:12.703849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:12.704147 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:12.704148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:12.704187 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:12.704716 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:12.704720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:12.706406 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":91,"threads_started":1}
W20260620 09:22:12.706611 13523 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:12.702175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:52267 (local address 127.4.59.62:42745)
0620 09:22:12.702241 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:12.702244 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:12.702367 (+ 123us) server_negotiation.cc:407] Connection header received
0620 09:22:12.702414 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:12.702414 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:12.702426 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:12.702475 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:12.702754 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:12.703245 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:12.703914 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:12.704086 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:12.704134 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:12.706531 (+ 2397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:52267: BlockingRecv error: recv got EOF from 127.4.59.3:52267 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 20 09:22:13 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:13 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:13.464676 13526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:13.460027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:47139)
0620 09:22:13.460182 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:13.460192 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:13.460250 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:13.460433 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:13.460434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:13.460595 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:13.460643 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:13.460644 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:13.461333 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:13.461334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:13.461866 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:13.461867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:13.462289 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:13.462290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:13.462349 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:13.462860 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:13.462865 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:13.464618 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
W20260620 09:22:13.464844 13527 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:13.460116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:47139 (local address 127.4.59.62:42745)
0620 09:22:13.460283 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:13.460285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:13.460294 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:22:13.460325 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:13.460326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:13.460336 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:13.460420 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:13.460732 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:13.461253 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:13.461963 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:13.462158 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:13.462220 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:13.464760 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:47139: BlockingRecv error: recv got EOF from 127.4.59.1:47139 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":116,"threads_started":1}
Jun 20 09:22:13 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:13 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:13.593529 13528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:13.585555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:58503)
0620 09:22:13.585689 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:13.585708 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:13.585770 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:13.585921 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:13.585921 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:13.586076 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:13.586124 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:13.586126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:13.586789 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:13.586790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:13.587311 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:13.587313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:13.587674 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:13.587675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:13.587710 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:13.591694 (+ 3984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:13.591700 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:13.593461 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
W20260620 09:22:13.593600 13527 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:13.585633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:58503 (local address 127.4.59.62:42745)
0620 09:22:13.585715 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:13.585718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:13.585755 (+ 37us) server_negotiation.cc:407] Connection header received
0620 09:22:13.585837 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:13.585838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:13.585848 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:13.585903 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:13.586207 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:13.586718 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:13.587407 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:13.587605 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:13.587668 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:13.593554 (+ 5886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:58503: BlockingRecv error: recv got EOF from 127.4.59.2:58503 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 20 09:22:13 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:13 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:13.711306 13529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:13.706887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:56055)
0620 09:22:13.707025 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:13.707035 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:13.707091 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:13.707226 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:13.707227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:13.707390 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:13.707438 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:13.707439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:13.708046 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:13.708047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:13.708558 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:13.708559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:13.708908 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:13.708909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:13.708946 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:13.709438 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:13.709443 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:13.711248 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
W20260620 09:22:13.711432 13527 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:13.706939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:56055 (local address 127.4.59.62:42745)
0620 09:22:13.707010 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:13.707012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:13.707092 (+ 80us) server_negotiation.cc:407] Connection header received
0620 09:22:13.707151 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:13.707152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:13.707164 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:13.707208 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:13.707500 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:13.707982 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:13.708645 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:13.708839 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:13.708902 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:13.711332 (+ 2430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:56055: BlockingRecv error: recv got EOF from 127.4.59.3:56055 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 20 09:22:14 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:14 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:14.470279 13530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:14.465218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:48403)
0620 09:22:14.465369 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:14.465380 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:14.465450 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:14.465590 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:14.465591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:14.465760 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:14.465809 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:14.465811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:14.466832 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:14.466833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:14.467377 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:14.467379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:14.467826 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:14.467832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:14.467879 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:14.468387 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:14.468393 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:14.470179 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
W20260620 09:22:14.470329 13531 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:14.465325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:48403 (local address 127.4.59.62:42745)
0620 09:22:14.465456 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:14.465460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:14.465473 (+ 13us) server_negotiation.cc:407] Connection header received
0620 09:22:14.465497 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:14.465498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:14.465507 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:14.465575 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:14.465941 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:14.466744 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:14.467471 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:14.467656 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:14.467705 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:14.470257 (+ 2552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:48403: BlockingRecv error: recv got EOF from 127.4.59.1:48403 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
Jun 20 09:22:14 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:14 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:14.598788 13532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:14.594017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:54101)
0620 09:22:14.594170 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:14.594186 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:14.594241 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:14.594369 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:14.594370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:14.594532 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:14.594581 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:14.594582 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:14.595200 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:14.595200 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:14.596001 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:14.596004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:14.596369 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:14.596370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:14.596429 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:14.596904 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:14.596908 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:14.598731 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":66,"threads_started":1}
W20260620 09:22:14.598979 13531 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:14.594132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:54101 (local address 127.4.59.62:42745)
0620 09:22:14.594226 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:14.594228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:14.594240 (+ 12us) server_negotiation.cc:407] Connection header received
0620 09:22:14.594290 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:14.594290 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:14.594300 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:14.594356 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:14.594644 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:14.595131 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:14.596075 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:14.596284 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:14.596337 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:14.598897 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:54101: BlockingRecv error: recv got EOF from 127.4.59.2:54101 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 20 09:22:14 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:14 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:14.716531 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:14.711755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:47045)
0620 09:22:14.711884 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:14.711895 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:14.711961 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:14.712122 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:14.712123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:14.712277 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:14.712349 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:14.712351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:14.713092 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:14.713093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:14.713641 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:14.713643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:14.713986 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:14.713987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:14.714025 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:14.714624 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:14.714630 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:14.716461 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
W20260620 09:22:14.716588 13531 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:14.711808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:47045 (local address 127.4.59.62:42745)
0620 09:22:14.711878 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:14.711879 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:14.711961 (+ 82us) server_negotiation.cc:407] Connection header received
0620 09:22:14.712004 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:14.712005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:14.712014 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:14.712116 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:14.712483 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:14.713026 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:14.713713 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:14.713912 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:14.713978 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:14.716541 (+ 2563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:47045: BlockingRecv error: recv got EOF from 127.4.59.3:47045 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 20 09:22:15 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:15 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:15.475873 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:15.470794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:32777)
0620 09:22:15.470955 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:15.470966 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:15.471017 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:15.471158 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:15.471159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:15.471327 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:15.471376 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:15.471378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:15.472108 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:15.472109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:15.472683 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:15.472685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:15.473078 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:15.473079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:15.473130 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:15.473725 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:15.473730 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:15.475777 (+ 2047us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":109,"threads_started":1}
W20260620 09:22:15.475925 13535 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:15.470865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:32777 (local address 127.4.59.62:42745)
0620 09:22:15.470985 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:15.470987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:15.471001 (+ 14us) server_negotiation.cc:407] Connection header received
0620 09:22:15.471072 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:15.471072 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:15.471082 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:15.471145 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:15.471480 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:15.472011 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:15.472782 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:15.472983 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:15.473044 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:15.475854 (+ 2810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:32777: BlockingRecv error: recv got EOF from 127.4.59.1:32777 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
Jun 20 09:22:15 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:15 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:15.607296 13536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:15.599319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:55821)
0620 09:22:15.599476 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:15.599490 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:15.599565 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:15.599725 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:15.599726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:15.599904 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:15.599952 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:15.599953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:15.600847 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:15.600849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:15.601432 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:15.601433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:15.601816 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:15.601817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:15.601873 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:15.605346 (+ 3473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:15.605350 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:15.607196 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":78,"threads_started":1}
W20260620 09:22:15.607347 13535 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:15.599405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:55821 (local address 127.4.59.62:42745)
0620 09:22:15.599476 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:15.599479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:15.599562 (+ 83us) server_negotiation.cc:407] Connection header received
0620 09:22:15.599642 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:15.599642 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:15.599657 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:15.599720 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:15.600030 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:15.600748 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:15.601518 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:15.601731 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:15.601815 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:15.607282 (+ 5467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:55821: BlockingRecv error: recv got EOF from 127.4.59.2:55821 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 20 09:22:15 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:15 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:15.723722 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:15.717080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:41085)
0620 09:22:15.717248 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:15.717259 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:15.717331 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:15.717554 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:15.717555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:15.717778 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:15.717836 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:15.717838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:15.718879 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:15.718880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:15.719939 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:15.719942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:15.720479 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:15.720480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:15.720616 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:15.721183 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:15.721189 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:15.723635 (+ 2446us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":96,"threads_started":1}
W20260620 09:22:15.723839 13535 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:15.717129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:41085 (local address 127.4.59.62:42745)
0620 09:22:15.717193 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:15.717196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:15.717312 (+ 116us) server_negotiation.cc:407] Connection header received
0620 09:22:15.717429 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:15.717431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:15.717465 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:15.717540 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:15.717912 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:15.718785 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:15.720035 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:15.720384 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:15.720470 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:15.723757 (+ 3287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:41085: BlockingRecv error: recv got EOF from 127.4.59.3:41085 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 20 09:22:16 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:16 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:16.481138 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:16.476379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:53471)
0620 09:22:16.476529 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:16.476539 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:16.476598 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:16.476784 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:16.476785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:16.476940 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:16.476990 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:16.476991 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:16.477814 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:16.477815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:16.478372 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:16.478374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:16.478693 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:16.478693 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:16.478733 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:16.479151 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:16.479156 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:16.481039 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":71,"threads_started":1}
W20260620 09:22:16.481216 13539 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:16.476456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:53471 (local address 127.4.59.62:42745)
0620 09:22:16.476610 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:16.476612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:16.476643 (+ 31us) server_negotiation.cc:407] Connection header received
0620 09:22:16.476676 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:16.476676 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:16.476693 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:16.476758 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:16.477058 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:16.477720 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:16.478444 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:16.478624 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:16.478678 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:16.481123 (+ 2445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:53471: BlockingRecv error: recv got EOF from 127.4.59.1:53471 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
Jun 20 09:22:16 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:16 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:16.612687 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:16.607837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:43815)
0620 09:22:16.607991 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:16.608000 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:16.608053 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:16.608188 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:16.608189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:16.608397 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:16.608446 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:16.608448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:16.609080 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:16.609081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:16.609654 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:16.609657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:16.609977 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:16.609977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:16.610020 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:16.610443 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:16.610447 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:16.612553 (+ 2106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":67,"threads_started":1}
W20260620 09:22:16.612708 13539 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:16.607891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:43815 (local address 127.4.59.62:42745)
0620 09:22:16.607970 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:16.607974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:16.608051 (+ 77us) server_negotiation.cc:407] Connection header received
0620 09:22:16.608098 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:16.608099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:16.608108 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:16.608176 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:16.608504 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:16.609007 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:16.609732 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:16.609913 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:16.609962 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:16.612650 (+ 2688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:43815: BlockingRecv error: recv got EOF from 127.4.59.2:43815 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 20 09:22:16 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:16 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:16.729061 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:16.724341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:35033)
0620 09:22:16.724516 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:16.724528 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:16.724639 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:16.724789 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:16.724790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:16.724970 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:16.725014 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:16.725015 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:16.725722 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:16.725723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:16.726258 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:16.726260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:16.726640 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:16.726641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:16.726707 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:16.727202 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:16.727207 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:16.729005 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":85,"threads_started":1}
W20260620 09:22:16.729203 13539 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:16.724444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:35033 (local address 127.4.59.62:42745)
0620 09:22:16.724518 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:16.724520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:16.724618 (+ 98us) server_negotiation.cc:407] Connection header received
0620 09:22:16.724695 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:16.724696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:16.724711 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:16.724783 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:16.725106 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:16.725644 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:16.726350 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:16.726553 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:16.726639 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:16.729101 (+ 2462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:35033: BlockingRecv error: recv got EOF from 127.4.59.3:35033 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 20 09:22:17 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:17 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:17.486219 13542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:17.481628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:35509)
0620 09:22:17.481774 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:17.481784 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:17.481833 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:17.482032 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:17.482032 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:17.482195 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:17.482241 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:17.482242 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:17.482899 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:17.482900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:17.483431 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:17.483432 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:17.483759 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:17.483760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:17.483804 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:17.484341 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:17.484346 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:17.486112 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
W20260620 09:22:17.486271 13543 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:17.481774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:35509 (local address 127.4.59.62:42745)
0620 09:22:17.481912 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:17.481914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:17.481921 (+ 7us) server_negotiation.cc:407] Connection header received
0620 09:22:17.481942 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:17.481942 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:17.481952 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:17.482013 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:17.482307 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:17.482800 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:17.483510 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:17.483691 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:17.483755 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:17.486189 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:35509: BlockingRecv error: recv got EOF from 127.4.59.1:35509 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
Jun 20 09:22:17 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:17 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:17.618486 13544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:17.613262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:58183)
0620 09:22:17.613441 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:17.613452 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:17.613515 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:17.613666 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:17.613667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:17.613839 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:17.613885 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:17.613887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:17.614713 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:17.614714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:17.615322 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:17.615324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:17.615771 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:17.615772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:17.615872 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:17.616451 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:17.616456 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:17.618389 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":107,"threads_started":1}
W20260620 09:22:17.618507 13543 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:17.613298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:58183 (local address 127.4.59.62:42745)
0620 09:22:17.613383 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:17.613385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:17.613513 (+ 128us) server_negotiation.cc:407] Connection header received
0620 09:22:17.613590 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:17.613591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:17.613610 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:17.613655 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:17.613962 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:17.614619 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:17.615406 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:17.615694 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:17.615769 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:17.618452 (+ 2683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:58183: BlockingRecv error: recv got EOF from 127.4.59.2:58183 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 20 09:22:17 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:17 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:17.734081 13545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:17.729542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:57243)
0620 09:22:17.729707 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:17.729721 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:17.729803 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:17.729975 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:17.729976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:17.730144 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:17.730190 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:17.730192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:17.730864 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:17.730865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:17.731388 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:17.731389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:17.731750 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:17.731751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:17.731821 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:17.732264 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:17.732268 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:17.734024 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
W20260620 09:22:17.734334 13543 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:17.729655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:57243 (local address 127.4.59.62:42745)
0620 09:22:17.729737 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:17.729740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:17.729787 (+ 47us) server_negotiation.cc:407] Connection header received
0620 09:22:17.729897 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:17.729898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:17.729910 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:17.729969 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:17.730270 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:17.730776 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:17.731470 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:17.731663 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:17.731748 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:17.734244 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:57243: BlockingRecv error: recv got EOF from 127.4.59.3:57243 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 20 09:22:18 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:18 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:18.491786 13546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:18.486813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:58853)
0620 09:22:18.486990 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:18.487001 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:18.487057 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:18.487206 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:18.487207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:18.487380 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:18.487428 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:18.487430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:18.488321 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:18.488322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:18.488870 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:18.488873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:18.489275 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:18.489276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:18.489320 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:18.489753 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:18.489757 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:18.491726 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":86,"threads_started":1}
W20260620 09:22:18.491902 13547 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:18.486906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:58853 (local address 127.4.59.62:42745)
0620 09:22:18.487038 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:18.487040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:18.487048 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:22:18.487121 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:18.487121 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:18.487131 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:18.487187 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:18.487519 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:18.488212 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:18.488948 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:18.489204 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:18.489267 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:18.491817 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:58853: BlockingRecv error: recv got EOF from 127.4.59.1:58853 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
Jun 20 09:22:18 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:18 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:18.628985 13548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:18.619031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:40081)
0620 09:22:18.619173 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:18.619184 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:18.619246 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:18.619376 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:18.619377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:18.619546 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:18.619596 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:18.619598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:18.620207 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:18.620208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:18.620732 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:18.620733 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:18.621060 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:18.621061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:18.621151 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:18.626876 (+ 5725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:18.626888 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:18.628924 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
W20260620 09:22:18.629082 13547 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:18.619106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:40081 (local address 127.4.59.62:42745)
0620 09:22:18.619179 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:18.619180 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:18.619246 (+ 66us) server_negotiation.cc:407] Connection header received
0620 09:22:18.619294 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:18.619294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:18.619304 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:18.619361 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:18.619654 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:18.620155 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:18.620800 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:18.620977 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:18.621053 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:18.629017 (+ 7964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:40081: BlockingRecv error: recv got EOF from 127.4.59.2:40081 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 20 09:22:18 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:18 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:18.739305 13549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:18.734568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:52363)
0620 09:22:18.734747 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:18.734759 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:18.734827 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:18.735007 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:18.735008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:18.735186 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:18.735231 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:18.735232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:18.735911 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:18.735913 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:18.736452 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:18.736454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:18.736830 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:18.736831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:18.736891 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:18.737403 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:18.737408 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:18.739211 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":85,"threads_started":1}
W20260620 09:22:18.739343 13547 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:18.734657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:52363 (local address 127.4.59.62:42745)
0620 09:22:18.734739 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:18.734741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:18.734807 (+ 66us) server_negotiation.cc:407] Connection header received
0620 09:22:18.734889 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:18.734890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:18.734908 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:18.734999 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:18.735319 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:18.735831 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:18.736539 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:18.736727 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:18.736832 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:18.739283 (+ 2451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:52363: BlockingRecv error: recv got EOF from 127.4.59.3:52363 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 20 09:22:19 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:19 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:19.497139 13550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:19.492293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:42199)
0620 09:22:19.492556 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:19.492566 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:19.492626 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:19.492781 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:19.492782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:19.492940 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:19.492989 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:19.492990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:19.493660 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:19.493661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:19.494210 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:19.494212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:19.494576 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:19.494577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:19.494615 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:19.495056 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:19.495061 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:19.497042 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":186,"threads_started":1}
W20260620 09:22:19.497221 13551 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:19.492481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:42199 (local address 127.4.59.62:42745)
0620 09:22:19.492590 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:19.492592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:19.492614 (+ 22us) server_negotiation.cc:407] Connection header received
0620 09:22:19.492676 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:19.492677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:19.492687 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:19.492737 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:19.493055 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:19.493586 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:19.494302 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:19.494503 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:19.494568 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:19.497122 (+ 2554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:42199: BlockingRecv error: recv got EOF from 127.4.59.1:42199 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":50,"threads_started":1}
Jun 20 09:22:19 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:19 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:19.634215 13552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:19.629541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:53349)
0620 09:22:19.629705 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:19.629715 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:19.629772 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:19.629903 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:19.629903 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:19.630067 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:19.630115 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:19.630117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:19.630769 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:19.630769 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:19.631309 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:19.631311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:19.631634 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:19.631635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:19.631675 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:19.632154 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:19.632164 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:19.634075 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
W20260620 09:22:19.634318 13551 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:19.629648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:53349 (local address 127.4.59.62:42745)
0620 09:22:19.629702 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:19.629704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:19.629759 (+ 55us) server_negotiation.cc:407] Connection header received
0620 09:22:19.629819 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:19.629819 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:19.629828 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:19.629884 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:19.630203 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:19.630701 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:19.631387 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:19.631570 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:19.631626 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:19.634227 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:53349: BlockingRecv error: recv got EOF from 127.4.59.2:53349 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Jun 20 09:22:19 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:19 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:19.746246 13553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:19.739928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:42071)
0620 09:22:19.740088 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:19.740104 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:19.740391 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:19.740675 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:19.740676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:19.740996 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:19.741070 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:19.741075 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:19.741905 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:19.741906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:19.742436 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:19.742438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:19.742811 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:19.742812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:19.742872 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:19.743555 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:19.743561 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:19.746185 (+ 2624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
W20260620 09:22:19.746531 13551 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:19.740082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:42071 (local address 127.4.59.62:42745)
0620 09:22:19.740280 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:19.740283 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:19.740297 (+ 14us) server_negotiation.cc:407] Connection header received
0620 09:22:19.740576 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:19.740577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:19.740596 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:19.740665 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:19.741284 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:19.741802 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:19.742540 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:19.742730 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:19.742802 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:19.746446 (+ 3644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:42071: BlockingRecv error: recv got EOF from 127.4.59.3:42071 (error 108)
Metrics: {"server-negotiator.queue_time_us":148}
Jun 20 09:22:20 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:20 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:20.502192 13554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:20.497604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:39287)
0620 09:22:20.497759 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:20.497769 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:20.497819 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:20.497956 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:20.497957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:20.498116 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:20.498164 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:20.498165 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:20.498798 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:20.498799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:20.499309 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:20.499311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:20.499614 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:20.499615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:20.499652 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:20.500100 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:20.500105 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:20.502076 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
W20260620 09:22:20.502218 13555 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:20.497673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:39287 (local address 127.4.59.62:42745)
0620 09:22:20.497799 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:20.497801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:20.497811 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:22:20.497876 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:20.497877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:20.497886 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:20.497936 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:20.498234 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:20.498733 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:20.499377 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:20.499555 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:20.499602 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:20.502151 (+ 2549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:39287: BlockingRecv error: recv got EOF from 127.4.59.1:39287 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
Jun 20 09:22:20 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:20 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:20.639424 13556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:20.634743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:48559)
0620 09:22:20.634872 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:20.634883 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:20.634950 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:20.635217 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:20.635218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:20.635390 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:20.635438 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:20.635439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:20.636137 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:20.636138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:20.636681 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:20.636683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:20.636997 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:20.636998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:20.637035 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:20.637594 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:20.637599 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:20.639368 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
W20260620 09:22:20.639561 13555 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:20.634788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:48559 (local address 127.4.59.62:42745)
0620 09:22:20.634831 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:20.634834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:20.634931 (+ 97us) server_negotiation.cc:407] Connection header received
0620 09:22:20.635008 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:20.635009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:20.635022 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:20.635078 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:20.635507 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:20.636059 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:20.636759 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:20.636935 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:20.636984 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:20.639471 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:48559: BlockingRecv error: recv got EOF from 127.4.59.2:48559 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Jun 20 09:22:20 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:20 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:20.751500 13557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:20.746695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:34655)
0620 09:22:20.746897 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:20.746907 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:20.746965 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:20.747138 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:20.747139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:20.747532 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:20.747585 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:20.747587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:20.748281 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:20.748282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:20.748806 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:20.748808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:20.749304 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:20.749305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:20.749348 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:20.749801 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:20.749805 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:20.751443 (+ 1638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":121,"threads_started":1}
W20260620 09:22:20.751648 13555 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:20.746801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:34655 (local address 127.4.59.62:42745)
0620 09:22:20.746873 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:20.746875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:20.746964 (+ 89us) server_negotiation.cc:407] Connection header received
0620 09:22:20.747008 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:20.747009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:20.747020 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:20.747095 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:20.747689 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:20.748191 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:20.748894 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:20.749078 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:20.749188 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:20.751561 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:34655: BlockingRecv error: recv got EOF from 127.4.59.3:34655 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 20 09:22:21 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:21 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:21.507736 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:21.502664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:59247)
0620 09:22:21.502969 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:21.502981 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:21.503076 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:21.503278 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:21.503280 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:21.503493 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:21.503550 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:21.503553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:21.504184 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:21.504185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:21.504778 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:21.504790 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:21.505260 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:21.505261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:21.505319 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:21.505876 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:21.505880 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:21.507666 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":139,"threads_started":1}
W20260620 09:22:21.507836 13559 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:21.502767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:59247 (local address 127.4.59.62:42745)
0620 09:22:21.502889 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:21.502891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:21.503058 (+ 167us) server_negotiation.cc:407] Connection header received
0620 09:22:21.503148 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:21.503149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:21.503163 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:21.503257 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:21.503625 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:21.504118 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:21.504872 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:21.505118 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:21.505244 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:21.507762 (+ 2518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:59247: BlockingRecv error: recv got EOF from 127.4.59.1:59247 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":46,"threads_started":1}
Jun 20 09:22:21 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:21 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:21.644915 13560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:21.639920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:35195)
0620 09:22:21.640096 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:21.640108 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:21.640180 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:21.640312 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:21.640312 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:21.640495 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:21.640541 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:21.640543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:21.641396 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:21.641397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:21.641968 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:21.641971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:21.642364 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:21.642365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:21.642438 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:21.642883 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:21.642888 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:21.644810 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":90,"threads_started":1}
W20260620 09:22:21.644950 13559 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:21.640005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:35195 (local address 127.4.59.62:42745)
0620 09:22:21.640081 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:21.640083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:21.640172 (+ 89us) server_negotiation.cc:407] Connection header received
0620 09:22:21.640246 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:21.640247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:21.640258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:21.640307 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:21.640620 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:21.641116 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:21.642059 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:21.642281 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:21.642355 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:21.644885 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:35195: BlockingRecv error: recv got EOF from 127.4.59.2:35195 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 20 09:22:21 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:21 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:21.756685 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:21.752052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:39247)
0620 09:22:21.752201 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:21.752213 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:21.752282 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:21.752445 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:21.752446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:21.752629 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:21.752674 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:21.752675 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:21.753393 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:21.753394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:21.753933 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:21.753934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:21.754290 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:21.754291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:21.754345 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:21.754860 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:21.754865 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:21.756574 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
W20260620 09:22:21.756718 13559 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:21.752074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:39247 (local address 127.4.59.62:42745)
0620 09:22:21.752121 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:21.752123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:21.752264 (+ 141us) server_negotiation.cc:407] Connection header received
0620 09:22:21.752340 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:21.752341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:21.752357 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:21.752433 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:21.752755 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:21.753314 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:21.754017 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:21.754207 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:21.754278 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:21.756651 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:39247: BlockingRecv error: recv got EOF from 127.4.59.3:39247 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 20 09:22:22 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:22 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:22.513216 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:22.508200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:47531)
0620 09:22:22.508332 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:22.508342 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:22.508389 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:22.508598 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:22.508599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:22.508760 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:22.508809 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:22.508810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:22.509534 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:22.509536 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:22.510080 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:22.510082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:22.510504 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:22.510505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:22.510551 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:22.511025 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:22.511030 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:22.513080 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
W20260620 09:22:22.513253 13563 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:22.508316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:47531 (local address 127.4.59.62:42745)
0620 09:22:22.508455 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:22.508457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:22.508473 (+ 16us) server_negotiation.cc:407] Connection header received
0620 09:22:22.508494 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:22.508495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:22.508510 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:22.508575 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:22.508876 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:22.509444 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:22.510210 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:22.510415 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:22.510494 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:22.513178 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:47531: BlockingRecv error: recv got EOF from 127.4.59.1:47531 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
Jun 20 09:22:22 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:22 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:22.650126 13564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:22.645445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:39283)
0620 09:22:22.645654 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:22.645665 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:22.645748 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:22.645893 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:22.645894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:22.646150 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:22.646200 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:22.646203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:22.646876 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:22.646877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:22.647419 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:22.647421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:22.647754 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:22.647755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:22.647805 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:22.648248 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:22.648254 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:22.650063 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":65,"threads_started":1}
W20260620 09:22:22.650231 13563 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:22.645546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:39283 (local address 127.4.59.62:42745)
0620 09:22:22.645612 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:22.645614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:22.645751 (+ 137us) server_negotiation.cc:407] Connection header received
0620 09:22:22.645808 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:22.645808 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:22.645822 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:22.645872 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:22.646293 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:22.646790 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:22.647497 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:22.647683 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:22.647735 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:22.650160 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:39283: BlockingRecv error: recv got EOF from 127.4.59.2:39283 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 20 09:22:22 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:22 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:22.762115 13565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:22.757247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:40347)
0620 09:22:22.757396 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:22.757407 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:22.757473 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:22.757642 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:22.757643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:22.757802 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:22.757860 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:22.757862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:22.758673 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:22.758675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:22.759213 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:22.759215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:22.759729 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:22.759730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:22.759780 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:22.760302 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:22.760307 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:22.762046 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
W20260620 09:22:22.762233 13563 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:22.757293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:40347 (local address 127.4.59.62:42745)
0620 09:22:22.757371 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:22.757376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:22.757477 (+ 101us) server_negotiation.cc:407] Connection header received
0620 09:22:22.757539 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:22.757540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:22.757550 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:22.757593 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:22.757947 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:22.758437 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:22.759289 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:22.759474 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:22.759708 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:22.762148 (+ 2440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:40347: BlockingRecv error: recv got EOF from 127.4.59.3:40347 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 20 09:22:23 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:23 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:23.518287 13566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:23.513694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:53381)
0620 09:22:23.513836 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:23.513849 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:23.513921 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:23.514079 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:23.514080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:23.514242 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:23.514290 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:23.514292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:23.514923 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:23.514924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:23.515470 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:23.515471 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:23.515843 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:23.515844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:23.515889 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:23.516375 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:23.516379 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:23.518221 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":65,"threads_started":1}
W20260620 09:22:23.518388 13567 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:23.513746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:53381 (local address 127.4.59.62:42745)
0620 09:22:23.513872 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:23.513873 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:23.513904 (+ 31us) server_negotiation.cc:407] Connection header received
0620 09:22:23.513970 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:23.513971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:23.513980 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:23.514036 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:23.514362 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:23.514853 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:23.515569 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:23.515769 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:23.515831 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:23.518312 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:53381: BlockingRecv error: recv got EOF from 127.4.59.1:53381 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
Jun 20 09:22:23 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:23 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:23.660125 13568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:23.650603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:49255)
0620 09:22:23.650799 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:23.650811 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:23.650874 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:23.651012 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:23.651013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:23.651175 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:23.651223 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:23.651225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:23.651841 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:23.651842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:23.652433 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:23.652435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:23.652806 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:23.652807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:23.652879 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:23.658274 (+ 5395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:23.658278 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:23.660020 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":111,"threads_started":1}
W20260620 09:22:23.660166 13567 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:23.650687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:49255 (local address 127.4.59.62:42745)
0620 09:22:23.650756 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:23.650759 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:23.650875 (+ 116us) server_negotiation.cc:407] Connection header received
0620 09:22:23.650937 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:23.650938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:23.650949 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:23.650991 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:23.651285 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:23.651772 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:23.652512 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:23.652720 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:23.652805 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:23.660103 (+ 7298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:49255: BlockingRecv error: recv got EOF from 127.4.59.2:49255 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 20 09:22:23 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:23 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.3@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:23.767227 13569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:23.762652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.3:49271)
0620 09:22:23.762838 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:23.762849 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:23.762942 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:23.763112 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:23.763112 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:23.763289 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:23.763335 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:23.763337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:23.764016 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:23.764018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:23.764549 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:23.764551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:23.764919 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:23.764920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:23.764981 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:23.765495 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:23.765501 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:23.767158 (+ 1657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":100,"threads_started":1}
W20260620 09:22:23.767293 13567 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:23.762749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:49271 (local address 127.4.59.62:42745)
0620 09:22:23.762835 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:23.762837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:23.762923 (+ 86us) server_negotiation.cc:407] Connection header received
0620 09:22:23.762997 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:23.762998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:23.763012 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:23.763103 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:23.763417 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:23.763934 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:23.764636 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:23.764832 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:23.764917 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:23.767247 (+ 2330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.3:49271: BlockingRecv error: recv got EOF from 127.4.59.3:49271 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 20 09:22:24 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:24 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.1@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:24.523422 13570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:24.518790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.1:54971)
0620 09:22:24.518946 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:24.518957 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:24.519013 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:24.519145 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:24.519145 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:24.519311 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:24.519393 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:24.519394 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:24.520105 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:24.520110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:24.520814 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:24.520817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:24.521204 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:24.521205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:24.521264 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:24.521681 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:24.521685 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:24.523367 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
W20260620 09:22:24.523573 13571 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:24.518883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:54971 (local address 127.4.59.62:42745)
0620 09:22:24.519011 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:24.519013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:24.519022 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:22:24.519065 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:24.519065 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:24.519075 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:24.519125 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:24.519458 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:24.519973 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:24.520894 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:24.521096 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:24.521190 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:24.523489 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.1:54971: BlockingRecv error: recv got EOF from 127.4.59.1:54971 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
Jun 20 09:22:24 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 09:22:24 dist-test-slave-414w krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.59.2@KRBTEST.COM for oryx/127.4.59.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 09:22:24.665282 13572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:24.660601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42745 (local address 127.4.59.2:45945)
0620 09:22:24.660764 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:24.660777 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:24.660867 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:24.661016 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:24.661017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:24.661254 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:24.661307 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:24.661309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:24.661977 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:24.661978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:24.662559 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:24.662561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:24.662928 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:24.662929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:24.662987 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:24.663442 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:24.663447 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:24.665225 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.59.62:42745: Server oryx/127.4.59.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
W20260620 09:22:24.665397 13571 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:24.660679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:45945 (local address 127.4.59.62:42745)
0620 09:22:24.660762 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:24.660764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:24.660848 (+ 84us) server_negotiation.cc:407] Connection header received
0620 09:22:24.660926 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:24.660927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:24.660937 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:24.661011 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:24.661392 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:24.661894 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:24.662651 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:24.662854 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:24.662930 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:24.665322 (+ 2392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.59.2:45945: BlockingRecv error: recv got EOF from 127.4.59.2:45945 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 09:22:24.680801 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 13096
I20260620 09:22:24.685302 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 13248
I20260620 09:22:24.689597 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 13380
I20260620 09:22:24.693828 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 13026
2026-06-20T09:22:24Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17394 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-20T09:22:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:22:24Z Disabled control of system clock
I20260620 09:22:24.720674 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:33347
--webserver_interface=127.4.59.62
--webserver_port=0
--builtin_ntp_servers=127.4.59.20:40847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:33347
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 09:22:24.801949 13576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:24.802142 13576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:24.802213 13576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:24.803375 13576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:22:24.803413 13576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:24.803426 13576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:24.803463 13576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:22:24.803490 13576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:22:24.804796 13576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:33347
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:33347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:24.805038 13576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:24.805371 13576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:24.810360 13581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:24.810403 13576 server_base.cc:1034] running on GCE node
W20260620 09:22:24.810315 13582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:24.810348 13584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:24.810737 13576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:24.811041 13576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:24.812199 13576 hybrid_clock.cc:648] HybridClock initialized: now 1781947344812194 us; error 36 us; skew 500 ppm
I20260620 09:22:24.813407 13576 webserver.cc:460] Webserver started at http://127.4.59.62:33351/ using document root <none> and password file <none>
I20260620 09:22:24.813630 13576 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:24.813704 13576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:24.813866 13576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:24.814885 13576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "e5735e6d232a4220b2afbb0aacec9452"
format_stamp: "Formatted at 2026-06-20 09:22:24 on dist-test-slave-414w"
server_key: "415b3fe22e2f18f9fd56da846749c03f"
server_key_iv: "6a0edf4d9dd4b9aa09008ab7341cfb94"
server_key_version: "encryptionkey@0"
I20260620 09:22:24.815248 13576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "e5735e6d232a4220b2afbb0aacec9452"
format_stamp: "Formatted at 2026-06-20 09:22:24 on dist-test-slave-414w"
server_key: "415b3fe22e2f18f9fd56da846749c03f"
server_key_iv: "6a0edf4d9dd4b9aa09008ab7341cfb94"
server_key_version: "encryptionkey@0"
I20260620 09:22:24.816520 13576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:22:24.817298 13590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:24.817476 13576 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:22:24.817571 13576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "e5735e6d232a4220b2afbb0aacec9452"
format_stamp: "Formatted at 2026-06-20 09:22:24 on dist-test-slave-414w"
server_key: "415b3fe22e2f18f9fd56da846749c03f"
server_key_iv: "6a0edf4d9dd4b9aa09008ab7341cfb94"
server_key_version: "encryptionkey@0"
I20260620 09:22:24.817656 13576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:24.825227 13576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:24.825480 13576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:24.825626 13576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:24.829324 13576 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:33347
I20260620 09:22:24.829357 13642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:33347 every 8 connection(s)
I20260620 09:22:24.829658 13576 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:22:24.830287 13643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:24.832552 13643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452: Bootstrap starting.
I20260620 09:22:24.833053 13643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:24.833361 13643 log.cc:824] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452: Log is configured to *not* fsync() on all Append() calls
I20260620 09:22:24.833926 13643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452: No bootstrap required, opened a new log
I20260620 09:22:24.834604 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 13576
I20260620 09:22:24.834709 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:22:24.834923 4332 external_mini_cluster.cc:1428] Setting key 6b7115c8040532d3d77cf0ae4d63ea15
I20260620 09:22:24.835933 13643 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5735e6d232a4220b2afbb0aacec9452" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33347 } }
I20260620 09:22:24.836090 13643 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:24.836125 13643 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5735e6d232a4220b2afbb0aacec9452, State: Initialized, Role: FOLLOWER
I20260620 09:22:24.836270 13643 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5735e6d232a4220b2afbb0aacec9452" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33347 } }
I20260620 09:22:24.836370 13643 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:22:24.836401 13643 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:22:24.836437 13643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:24.837119 13643 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5735e6d232a4220b2afbb0aacec9452" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33347 } }
I20260620 09:22:24.837248 13643 leader_election.cc:302] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5735e6d232a4220b2afbb0aacec9452; no voters:
I20260620 09:22:24.837396 13643 leader_election.cc:288] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:22:24.837462 13648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:22:24.837604 13648 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [term 1 LEADER]: Becoming Leader. State: Replica: e5735e6d232a4220b2afbb0aacec9452, State: Running, Role: LEADER
I20260620 09:22:24.837615 13643 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:22:24.837755 13648 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5735e6d232a4220b2afbb0aacec9452" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33347 } }
I20260620 09:22:24.838076 13650 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5735e6d232a4220b2afbb0aacec9452" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5735e6d232a4220b2afbb0aacec9452" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33347 } } }
I20260620 09:22:24.838146 13650 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [sys.catalog]: This master's current role is: LEADER
I20260620 09:22:24.838454 13655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:22:24.838941 13655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:22:24.839260 13649 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5735e6d232a4220b2afbb0aacec9452. Latest consensus state: current_term: 1 leader_uuid: "e5735e6d232a4220b2afbb0aacec9452" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5735e6d232a4220b2afbb0aacec9452" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33347 } } }
I20260620 09:22:24.839618 13649 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452 [sys.catalog]: This master's current role is: LEADER
I20260620 09:22:24.840561 13655 catalog_manager.cc:1347] Generated new cluster ID: 7d5e879f82f34a37a5b2caa88b3ff625
I20260620 09:22:24.840612 13655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:22:24.842072 13647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:24.835404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49256 (local address 127.4.59.62:33347)
0620 09:22:24.835631 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:24.835640 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:24.835672 (+ 32us) server_negotiation.cc:407] Connection header received
0620 09:22:24.835894 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:24.835902 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:24.835974 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:24.836115 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:24.836389 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:24.838162 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:24.839576 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:24.840162 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:24.840276 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:24.841416 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:24.841420 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:24.841420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:22:24.841610 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:24.841647 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:22:24.841661 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:24.841731 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:24.841798 (+ 67us) server_negotiation.cc:299] Negotiation successful
0620 09:22:24.841878 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":91,"threads_started":1}
I20260620 09:22:24.867985 13655 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:22:24.868711 13655 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:22:24.876796 13655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5735e6d232a4220b2afbb0aacec9452: Generated new TSK 0
I20260620 09:22:24.876981 13655 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:22:24.886250 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:33347
--builtin_ntp_servers=127.4.59.20:40847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260620 09:22:24.965221 13667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:24.965400 13667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:24.965420 13667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:24.966492 13667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:24.966528 13667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:24.966552 13667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:22:24.967818 13667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:33347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:24.967989 13667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:24.968209 13667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:24.972911 13675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:24.972919 13672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:24.973101 13667 server_base.cc:1034] running on GCE node
W20260620 09:22:24.972959 13673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:24.973351 13667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:24.973598 13667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:24.974757 13667 hybrid_clock.cc:648] HybridClock initialized: now 1781947344974742 us; error 26 us; skew 500 ppm
I20260620 09:22:24.975929 13667 webserver.cc:460] Webserver started at http://127.4.59.1:41191/ using document root <none> and password file <none>
I20260620 09:22:24.976171 13667 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:24.976248 13667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:24.976397 13667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:24.977461 13667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "f40b9bc132b844be856f3b134fa4583b"
format_stamp: "Formatted at 2026-06-20 09:22:24 on dist-test-slave-414w"
server_key: "e7a97b15d9876ff38a7263631b1251e5"
server_key_iv: "6837be61d7cbed7408844abb13a0e6e1"
server_key_version: "encryptionkey@0"
I20260620 09:22:24.977797 13667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "f40b9bc132b844be856f3b134fa4583b"
format_stamp: "Formatted at 2026-06-20 09:22:24 on dist-test-slave-414w"
server_key: "e7a97b15d9876ff38a7263631b1251e5"
server_key_iv: "6837be61d7cbed7408844abb13a0e6e1"
server_key_version: "encryptionkey@0"
I20260620 09:22:24.979020 13667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:22:24.979658 13681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:24.979848 13667 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:22:24.979944 13667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "f40b9bc132b844be856f3b134fa4583b"
format_stamp: "Formatted at 2026-06-20 09:22:24 on dist-test-slave-414w"
server_key: "e7a97b15d9876ff38a7263631b1251e5"
server_key_iv: "6837be61d7cbed7408844abb13a0e6e1"
server_key_version: "encryptionkey@0"
I20260620 09:22:24.980019 13667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:24.999500 13667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:24.999792 13667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:24.999940 13667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:25.000142 13667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:22:25.000454 13667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:25.000509 13667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.000548 13667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:25.000578 13667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.006222 13667 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:33313
I20260620 09:22:25.006276 13794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:33313 every 8 connection(s)
I20260620 09:22:25.006567 13667 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:22:25.010696 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 13667
I20260620 09:22:25.010766 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:22:25.010957 4332 external_mini_cluster.cc:1428] Setting key cd83513ff3ad45d9a058494931387bcf
I20260620 09:22:25.011487 13647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:25.007277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:34493 (local address 127.4.59.62:33347)
0620 09:22:25.007359 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:25.007361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:25.007543 (+ 182us) server_negotiation.cc:407] Connection header received
0620 09:22:25.007998 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:25.007999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:25.008011 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:25.008056 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:25.008636 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:25.009114 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:25.010101 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:25.010282 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:25.010342 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:25.011206 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:25.011207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:25.011207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:22:25.011213 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:25.011226 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:22:25.011229 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:25.011288 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:25.011382 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 09:22:25.011410 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 09:22:25.011696 13797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:25.007229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:33347 (local address 127.4.59.1:34493)
0620 09:22:25.007430 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:25.007461 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:25.007922 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:25.008111 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:25.008115 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:25.008122 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:25.008572 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:25.008576 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:25.009235 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:25.009236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:25.010026 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:25.010028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:25.010356 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:25.010357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:25.010460 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:25.011038 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:22:25.011038 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:25.011074 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:22:25.011099 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:25.011302 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:25.011308 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:25.011311 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:22:25.011380 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 09:22:25.011477 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260620 09:22:25.011854 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:33347
--builtin_ntp_servers=127.4.59.20:40847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260620 09:22:25.012133 13795 heartbeater.cc:347] Connected to a master server at 127.4.59.62:33347
I20260620 09:22:25.012233 13795 heartbeater.cc:464] Registering TS with master...
I20260620 09:22:25.012410 13795 heartbeater.cc:511] Master 127.4.59.62:33347 requested a full tablet report, sending...
I20260620 09:22:25.012859 13607 ts_manager.cc:194] Registered new tserver with Master: f40b9bc132b844be856f3b134fa4583b (127.4.59.1:33313)
I20260620 09:22:25.013490 13607 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.59.1:34493
W20260620 09:22:25.092483 13798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:25.092746 13798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:25.092778 13798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:25.093973 13798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:25.094043 13798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:25.094086 13798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:22:25.095453 13798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:33347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:25.095717 13798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:25.096019 13798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:25.100991 13803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:25.101099 13798 server_base.cc:1034] running on GCE node
W20260620 09:22:25.101008 13804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:25.101007 13806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:25.101516 13798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:25.101763 13798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:25.102926 13798 hybrid_clock.cc:648] HybridClock initialized: now 1781947345102913 us; error 30 us; skew 500 ppm
I20260620 09:22:25.104341 13798 webserver.cc:460] Webserver started at http://127.4.59.2:36263/ using document root <none> and password file <none>
I20260620 09:22:25.104591 13798 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:25.104640 13798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:25.104771 13798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:25.105917 13798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "fb51d6dffe29405d93d0a7e95f5caae3"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "8a2b0e045b1dac0ee4113ae852586b1c"
server_key_iv: "eef38a71184b840473ce8a62d09e858d"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.106264 13798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "fb51d6dffe29405d93d0a7e95f5caae3"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "8a2b0e045b1dac0ee4113ae852586b1c"
server_key_iv: "eef38a71184b840473ce8a62d09e858d"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.107540 13798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 09:22:25.108314 13812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.108503 13798 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:22:25.108590 13798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "fb51d6dffe29405d93d0a7e95f5caae3"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "8a2b0e045b1dac0ee4113ae852586b1c"
server_key_iv: "eef38a71184b840473ce8a62d09e858d"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.108662 13798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:25.129648 13798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:25.129954 13798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:25.130102 13798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:25.130321 13798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:22:25.130643 13798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:25.130698 13798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.130739 13798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:25.130766 13798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.136514 13798 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:35923
I20260620 09:22:25.136559 13925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:35923 every 8 connection(s)
I20260620 09:22:25.136924 13798 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:22:25.142122 13647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:25.137596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:53043 (local address 127.4.59.62:33347)
0620 09:22:25.137698 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:25.137700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:25.137937 (+ 237us) server_negotiation.cc:407] Connection header received
0620 09:22:25.138508 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:25.138509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:25.138520 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:25.138572 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:25.139337 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:25.139825 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:25.140787 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:25.141042 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:25.141103 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:25.141900 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:25.141901 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:25.141901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:22:25.141905 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:25.141917 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:22:25.141921 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:25.141965 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:25.142045 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 09:22:25.142066 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 09:22:25.142338 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:25.137560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:33347 (local address 127.4.59.2:53043)
0620 09:22:25.137777 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:25.137810 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:25.138397 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:25.138629 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:25.138632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:25.138640 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:25.139239 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:25.139245 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:25.139910 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:25.139911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:25.140714 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:25.140717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:25.141127 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:25.141128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:25.141308 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:25.141773 (+ 465us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:22:25.141774 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:25.141800 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:22:25.141821 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:25.141975 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:25.141979 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:25.141981 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:22:25.142031 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 09:22:25.142106 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":115,"threads_started":1}
I20260620 09:22:25.142992 13926 heartbeater.cc:347] Connected to a master server at 127.4.59.62:33347
I20260620 09:22:25.143086 13926 heartbeater.cc:464] Registering TS with master...
I20260620 09:22:25.143298 13926 heartbeater.cc:511] Master 127.4.59.62:33347 requested a full tablet report, sending...
I20260620 09:22:25.143668 13607 ts_manager.cc:194] Registered new tserver with Master: fb51d6dffe29405d93d0a7e95f5caae3 (127.4.59.2:35923)
I20260620 09:22:25.144059 13607 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.59.2:53043
I20260620 09:22:25.147045 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 13798
I20260620 09:22:25.147118 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:22:25.147325 4332 external_mini_cluster.cc:1428] Setting key a001242e71378624ce3b10c278724136
I20260620 09:22:25.148206 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:33347
--builtin_ntp_servers=127.4.59.20:40847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260620 09:22:25.226058 13929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:25.226234 13929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:25.226254 13929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:25.227360 13929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:25.227398 13929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:25.227422 13929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:22:25.228714 13929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:33347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:25.228927 13929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:25.229221 13929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:25.233786 13937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:25.233812 13935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:25.233783 13934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:25.233886 13929 server_base.cc:1034] running on GCE node
I20260620 09:22:25.234148 13929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:25.234391 13929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:25.235690 13929 hybrid_clock.cc:648] HybridClock initialized: now 1781947345235679 us; error 37 us; skew 500 ppm
I20260620 09:22:25.237248 13929 webserver.cc:460] Webserver started at http://127.4.59.3:41027/ using document root <none> and password file <none>
I20260620 09:22:25.237483 13929 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:25.237550 13929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:25.237681 13929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:25.238813 13929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "ac5711ba5fe742c68a4b084dd656bca3"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "21c696d447889c7c2e4972564fe29a87"
server_key_iv: "4665ed556a48fb661617a1a9033c7c6e"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.239179 13929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "ac5711ba5fe742c68a4b084dd656bca3"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "21c696d447889c7c2e4972564fe29a87"
server_key_iv: "4665ed556a48fb661617a1a9033c7c6e"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.240499 13929 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:22:25.241127 13943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.241330 13929 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:22:25.241396 13929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "ac5711ba5fe742c68a4b084dd656bca3"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "21c696d447889c7c2e4972564fe29a87"
server_key_iv: "4665ed556a48fb661617a1a9033c7c6e"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.241477 13929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:25.270124 13929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:25.270423 13929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:25.270573 13929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:25.270800 13929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:22:25.271124 13929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:25.271180 13929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.271230 13929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:25.271261 13929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.276734 13929 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:35235
I20260620 09:22:25.276827 14056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:35235 every 8 connection(s)
I20260620 09:22:25.277113 13929 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:22:25.282011 13647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:25.277853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:58787 (local address 127.4.59.62:33347)
0620 09:22:25.277901 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:25.277903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:25.278106 (+ 203us) server_negotiation.cc:407] Connection header received
0620 09:22:25.278551 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:25.278552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:25.278562 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:25.278609 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:25.279260 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:25.279735 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:25.280672 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:25.280859 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:25.280921 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:25.281806 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:25.281807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:25.281807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:22:25.281811 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:25.281822 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:22:25.281826 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:25.281868 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:25.281929 (+ 61us) server_negotiation.cc:299] Negotiation successful
0620 09:22:25.281951 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 09:22:25.282169 14059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:25.277832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:33347 (local address 127.4.59.3:58787)
0620 09:22:25.277996 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:25.278026 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:25.278474 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:25.278688 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:25.278691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:25.278699 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:25.279175 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:25.279179 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:25.279822 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:25.279823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:25.280574 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:25.280576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:25.280946 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:25.280947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:25.281060 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:25.281656 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:22:25.281656 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:25.281702 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:22:25.281724 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:25.281870 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:25.281872 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:25.281875 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:22:25.281919 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 09:22:25.281988 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260620 09:22:25.282671 14057 heartbeater.cc:347] Connected to a master server at 127.4.59.62:33347
I20260620 09:22:25.282788 14057 heartbeater.cc:464] Registering TS with master...
I20260620 09:22:25.282783 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 13929
I20260620 09:22:25.282861 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:22:25.282994 14057 heartbeater.cc:511] Master 127.4.59.62:33347 requested a full tablet report, sending...
I20260620 09:22:25.283061 4332 external_mini_cluster.cc:1428] Setting key 0becbcfe6da2b6560463587c65c8b0ad
I20260620 09:22:25.283367 13607 ts_manager.cc:194] Registered new tserver with Master: ac5711ba5fe742c68a4b084dd656bca3 (127.4.59.3:35235)
I20260620 09:22:25.283879 13607 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.59.3:58787
I20260620 09:22:25.283968 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 09:22:25.285645 13647 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:25.285229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49272 (local address 127.4.59.62:33347)
0620 09:22:25.285288 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:25.285290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:25.285394 (+ 104us) server_negotiation.cc:407] Connection header received
0620 09:22:25.285427 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:25.285427 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:25.285437 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:25.285470 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:25.285593 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49272: BlockingRecv error: recv got EOF from 127.0.0.1:49272 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 09:22:25.285702 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 13667
I20260620 09:22:25.290827 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 13798
I20260620 09:22:25.295428 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 13929
I20260620 09:22:25.299388 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 13576
2026-06-20T09:22:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (604 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-20T09:22:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:22:25Z Disabled control of system clock
I20260620 09:22:25.324869 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:44653
--webserver_interface=127.4.59.62
--webserver_port=0
--builtin_ntp_servers=127.4.59.20:41541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:44653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 09:22:25.403699 14068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:25.403895 14068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:25.403916 14068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:25.404953 14068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:22:25.404986 14068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:25.404999 14068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:25.405011 14068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:22:25.405023 14068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:22:25.406345 14068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:41541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:44653
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:44653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:25.406529 14068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:25.406742 14068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:25.411558 14074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:25.411659 14068 server_base.cc:1034] running on GCE node
W20260620 09:22:25.411577 14073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:25.411586 14076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:25.411979 14068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:25.412257 14068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:25.413393 14068 hybrid_clock.cc:648] HybridClock initialized: now 1781947345413368 us; error 39 us; skew 500 ppm
I20260620 09:22:25.414551 14068 webserver.cc:460] Webserver started at http://127.4.59.62:39989/ using document root <none> and password file <none>
I20260620 09:22:25.414785 14068 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:25.414832 14068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:25.415001 14068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:25.416074 14068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "b5d333208de74314901f901e28409054"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "0d7702b2989497987ce52b8a374a77ca"
server_key_iv: "5b8fd3c511f47f1abd027bb008013d85"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.416441 14068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "b5d333208de74314901f901e28409054"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "0d7702b2989497987ce52b8a374a77ca"
server_key_iv: "5b8fd3c511f47f1abd027bb008013d85"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.417896 14068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:22:25.418661 14082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.418852 14068 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:22:25.419028 14068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "b5d333208de74314901f901e28409054"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "0d7702b2989497987ce52b8a374a77ca"
server_key_iv: "5b8fd3c511f47f1abd027bb008013d85"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.419154 14068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:25.422706 14068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:25.422991 14068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:25.423141 14068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:25.427202 14068 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:44653
I20260620 09:22:25.427263 14134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:44653 every 8 connection(s)
I20260620 09:22:25.427600 14068 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:22:25.428331 14135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:25.428491 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 14068
I20260620 09:22:25.428592 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:22:25.428794 4332 external_mini_cluster.cc:1428] Setting key 275d2898b2bebdb256cf01a01d605de0
I20260620 09:22:25.431541 14138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:25.429494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36582 (local address 127.4.59.62:44653)
0620 09:22:25.429704 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:25.429715 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:25.429747 (+ 32us) server_negotiation.cc:407] Connection header received
0620 09:22:25.430161 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:25.430169 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:25.430222 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:25.430381 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:25.431117 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:25.431121 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:25.431121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:22:25.431125 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:25.431171 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:22:25.431186 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:25.431268 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:25.431300 (+ 32us) server_negotiation.cc:299] Negotiation successful
0620 09:22:25.431355 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
I20260620 09:22:25.432435 14135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054: Bootstrap starting.
I20260620 09:22:25.433022 14135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:25.433344 14135 log.cc:824] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054: Log is configured to *not* fsync() on all Append() calls
I20260620 09:22:25.434031 14135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054: No bootstrap required, opened a new log
I20260620 09:22:25.435570 14135 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d333208de74314901f901e28409054" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 44653 } }
I20260620 09:22:25.435724 14135 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:25.435786 14135 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5d333208de74314901f901e28409054, State: Initialized, Role: FOLLOWER
I20260620 09:22:25.435912 14135 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d333208de74314901f901e28409054" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 44653 } }
I20260620 09:22:25.436003 14135 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:22:25.436048 14135 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:22:25.436097 14135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:25.436607 14135 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d333208de74314901f901e28409054" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 44653 } }
I20260620 09:22:25.436720 14135 leader_election.cc:302] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5d333208de74314901f901e28409054; no voters:
I20260620 09:22:25.436980 14135 leader_election.cc:288] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:22:25.437314 14140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:22:25.437340 14135 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:22:25.437469 14140 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [term 1 LEADER]: Becoming Leader. State: Replica: b5d333208de74314901f901e28409054, State: Running, Role: LEADER
I20260620 09:22:25.437551 14140 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d333208de74314901f901e28409054" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 44653 } }
I20260620 09:22:25.438001 14140 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5d333208de74314901f901e28409054. Latest consensus state: current_term: 1 leader_uuid: "b5d333208de74314901f901e28409054" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d333208de74314901f901e28409054" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 44653 } } }
I20260620 09:22:25.438014 14141 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5d333208de74314901f901e28409054" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d333208de74314901f901e28409054" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 44653 } } }
I20260620 09:22:25.438117 14140 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [sys.catalog]: This master's current role is: LEADER
I20260620 09:22:25.438120 14141 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054 [sys.catalog]: This master's current role is: LEADER
I20260620 09:22:25.438844 14149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:22:25.439643 14149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:22:25.441215 14149 catalog_manager.cc:1347] Generated new cluster ID: d9d7294189d248d0940ce16f8c86c097
I20260620 09:22:25.441272 14149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:22:25.456874 14149 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:22:25.457052 14149 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:22:25.469213 14149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5d333208de74314901f901e28409054: Generated new TSK 0
I20260620 09:22:25.469467 14149 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:22:25.471853 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:44653
--builtin_ntp_servers=127.4.59.20:41541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 09:22:25.569334 14159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:25.569545 14159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:25.569578 14159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:25.570698 14159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:25.570761 14159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:25.570804 14159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:22:25.572113 14159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:41541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:44653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:25.572360 14159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:25.572638 14159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:25.578504 14167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:25.578486 14165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:25.578881 14159 server_base.cc:1034] running on GCE node
W20260620 09:22:25.578570 14164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:25.579232 14159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:25.579473 14159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:25.580623 14159 hybrid_clock.cc:648] HybridClock initialized: now 1781947345580607 us; error 33 us; skew 500 ppm
I20260620 09:22:25.581826 14159 webserver.cc:460] Webserver started at http://127.4.59.1:34765/ using document root <none> and password file <none>
I20260620 09:22:25.582046 14159 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:25.582086 14159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:25.582222 14159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:25.583201 14159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "f000f75c7ad241d99810c7c8c3b042e6"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "f6f88734b7d09a7376ca665a1ed16463"
server_key_iv: "0624c0c9185b65ef42df223c875a6bc0"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.583537 14159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "f000f75c7ad241d99810c7c8c3b042e6"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "f6f88734b7d09a7376ca665a1ed16463"
server_key_iv: "0624c0c9185b65ef42df223c875a6bc0"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.584803 14159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:22:25.585610 14173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.585801 14159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 09:22:25.585891 14159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "f000f75c7ad241d99810c7c8c3b042e6"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "f6f88734b7d09a7376ca665a1ed16463"
server_key_iv: "0624c0c9185b65ef42df223c875a6bc0"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.585975 14159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:25.589191 14159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:25.589428 14159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:25.589563 14159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:25.589763 14159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:22:25.590054 14159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:25.590106 14159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.590143 14159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:25.590169 14159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.595860 14159 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:38499
I20260620 09:22:25.595902 14286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:38499 every 8 connection(s)
I20260620 09:22:25.596185 14159 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:22:25.597349 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 14159
I20260620 09:22:25.597450 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:22:25.597640 4332 external_mini_cluster.cc:1428] Setting key dcd2ad1e9dfab0595ce04c7034fb4e49
I20260620 09:22:25.598649 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:44653
--builtin_ntp_servers=127.4.59.20:41541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260620 09:22:25.599287 14138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:25.596829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:33339 (local address 127.4.59.62:44653)
0620 09:22:25.596890 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:25.596891 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:25.597113 (+ 222us) server_negotiation.cc:407] Connection header received
0620 09:22:25.597698 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:25.597699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:25.597706 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:25.597769 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:25.599048 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:25.599049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:25.599050 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:22:25.599055 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:25.599069 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:22:25.599073 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:25.599133 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:25.599205 (+ 72us) server_negotiation.cc:299] Negotiation successful
0620 09:22:25.599219 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 09:22:25.599474 14289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:25.596808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:44653 (local address 127.4.59.1:33339)
0620 09:22:25.597005 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:25.597035 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:25.597595 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:25.597833 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:25.597837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:25.597846 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:25.598899 (+ 1053us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:22:25.598900 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:25.598937 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:22:25.598963 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:25.599131 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:25.599136 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:25.599139 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:22:25.599214 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 09:22:25.599269 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260620 09:22:25.599932 14287 heartbeater.cc:347] Connected to a master server at 127.4.59.62:44653
I20260620 09:22:25.600028 14287 heartbeater.cc:464] Registering TS with master...
I20260620 09:22:25.600171 14287 heartbeater.cc:511] Master 127.4.59.62:44653 requested a full tablet report, sending...
I20260620 09:22:25.600672 14099 ts_manager.cc:194] Registered new tserver with Master: f000f75c7ad241d99810c7c8c3b042e6 (127.4.59.1:38499)
W20260620 09:22:25.679347 14290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:25.679569 14290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:25.679601 14290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:25.680721 14290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:25.680787 14290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:25.680831 14290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:22:25.682189 14290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:41541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:44653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:25.682448 14290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:25.682737 14290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:25.687640 14295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:25.687659 14296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:25.687657 14298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:25.687816 14290 server_base.cc:1034] running on GCE node
I20260620 09:22:25.688092 14290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:25.688378 14290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:25.689517 14290 hybrid_clock.cc:648] HybridClock initialized: now 1781947345689507 us; error 27 us; skew 500 ppm
I20260620 09:22:25.690708 14290 webserver.cc:460] Webserver started at http://127.4.59.2:38847/ using document root <none> and password file <none>
I20260620 09:22:25.690958 14290 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:25.691006 14290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:25.691172 14290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:25.692163 14290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "4ebc0cd087d7496581e13b5894ee0b87"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "d84422accb5097c019e863e6291e4034"
server_key_iv: "08448ec68665f8901c8e039a4231b60e"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.692503 14290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "4ebc0cd087d7496581e13b5894ee0b87"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "d84422accb5097c019e863e6291e4034"
server_key_iv: "08448ec68665f8901c8e039a4231b60e"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.693934 14290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:22:25.694707 14304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.695034 14290 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:22:25.695106 14290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "4ebc0cd087d7496581e13b5894ee0b87"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "d84422accb5097c019e863e6291e4034"
server_key_iv: "08448ec68665f8901c8e039a4231b60e"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.695185 14290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:25.698583 14290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:25.698824 14290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:25.698966 14290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:25.699163 14290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:22:25.699447 14290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:25.699508 14290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.699549 14290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:25.699575 14290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.705185 14290 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:46557
I20260620 09:22:25.705248 14417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:46557 every 8 connection(s)
I20260620 09:22:25.705562 14290 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:22:25.708312 14138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:25.706224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:33995 (local address 127.4.59.62:44653)
0620 09:22:25.706292 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:25.706295 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:25.706514 (+ 219us) server_negotiation.cc:407] Connection header received
0620 09:22:25.706907 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:25.706908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:25.706914 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:25.706965 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:25.708099 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:25.708100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:25.708101 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:22:25.708105 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:25.708122 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:22:25.708126 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:25.708191 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:25.708238 (+ 47us) server_negotiation.cc:299] Negotiation successful
0620 09:22:25.708259 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 09:22:25.708508 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:25.706211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:44653 (local address 127.4.59.2:33995)
0620 09:22:25.706394 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:25.706424 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:25.706829 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:25.707012 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:25.707015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:25.707023 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:25.707947 (+ 924us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:22:25.707948 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:25.707976 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:22:25.708002 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:25.708190 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:25.708192 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:25.708194 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:22:25.708242 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 09:22:25.708285 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260620 09:22:25.708971 14418 heartbeater.cc:347] Connected to a master server at 127.4.59.62:44653
I20260620 09:22:25.709064 14418 heartbeater.cc:464] Registering TS with master...
I20260620 09:22:25.709221 14418 heartbeater.cc:511] Master 127.4.59.62:44653 requested a full tablet report, sending...
I20260620 09:22:25.709566 14099 ts_manager.cc:194] Registered new tserver with Master: 4ebc0cd087d7496581e13b5894ee0b87 (127.4.59.2:46557)
I20260620 09:22:25.713709 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 14290
I20260620 09:22:25.713814 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:22:25.714030 4332 external_mini_cluster.cc:1428] Setting key f26e0886e17abdea33c249cc03346a1e
I20260620 09:22:25.714880 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:44653
--builtin_ntp_servers=127.4.59.20:41541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 09:22:25.819672 14421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:25.819980 14421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:25.820020 14421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:25.821796 14421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:25.821861 14421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:25.821905 14421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:22:25.823807 14421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:41541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:44653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:25.824075 14421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:25.824366 14421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:25.829305 14427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:25.829309 14426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:25.829548 14421 server_base.cc:1034] running on GCE node
W20260620 09:22:25.829334 14429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:25.829852 14421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:25.830076 14421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:25.831238 14421 hybrid_clock.cc:648] HybridClock initialized: now 1781947345831203 us; error 48 us; skew 500 ppm
I20260620 09:22:25.832556 14421 webserver.cc:460] Webserver started at http://127.4.59.3:36119/ using document root <none> and password file <none>
I20260620 09:22:25.832794 14421 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:25.832841 14421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:25.833006 14421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:25.834115 14421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "911db58da6e645e29284e66861f90c6c"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "440fbc0cae33a6e75a8c418d714778d5"
server_key_iv: "1f220d8767d8b11c1e3742a22ada0114"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.834482 14421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "911db58da6e645e29284e66861f90c6c"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "440fbc0cae33a6e75a8c418d714778d5"
server_key_iv: "1f220d8767d8b11c1e3742a22ada0114"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.835739 14421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:22:25.836539 14435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.836735 14421 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:22:25.836841 14421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "911db58da6e645e29284e66861f90c6c"
format_stamp: "Formatted at 2026-06-20 09:22:25 on dist-test-slave-414w"
server_key: "440fbc0cae33a6e75a8c418d714778d5"
server_key_iv: "1f220d8767d8b11c1e3742a22ada0114"
server_key_version: "encryptionkey@0"
I20260620 09:22:25.836933 14421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:25.840538 14421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:25.840801 14421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:25.840942 14421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:25.841192 14421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:22:25.841482 14421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:25.841534 14421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.841570 14421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:25.841599 14421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:25.847259 14421 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:43623
I20260620 09:22:25.847308 14548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:43623 every 8 connection(s)
I20260620 09:22:25.847585 14421 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:22:25.849560 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 14421
I20260620 09:22:25.849650 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:22:25.849835 4332 external_mini_cluster.cc:1428] Setting key 6e25962684198ccd70a66ba75b6d52ff
I20260620 09:22:25.850687 14138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:25.848304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:49657 (local address 127.4.59.62:44653)
0620 09:22:25.848345 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:25.848347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:25.848589 (+ 242us) server_negotiation.cc:407] Connection header received
0620 09:22:25.849010 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:25.849011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:25.849016 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:25.849068 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:25.850413 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:25.850415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:25.850415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:22:25.850420 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:25.850436 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:22:25.850440 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:25.850500 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:25.850599 (+ 99us) server_negotiation.cc:299] Negotiation successful
0620 09:22:25.850613 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:22:25.850868 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:25.848320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:44653 (local address 127.4.59.3:49657)
0620 09:22:25.848477 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:25.848506 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:25.848930 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:25.849114 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:25.849117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:25.849124 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:25.850259 (+ 1135us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:22:25.850260 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:25.850297 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:22:25.850329 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:25.850507 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:25.850511 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:25.850514 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:22:25.850588 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 09:22:25.850647 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":82,"threads_started":1}
I20260620 09:22:25.851275 14549 heartbeater.cc:347] Connected to a master server at 127.4.59.62:44653
I20260620 09:22:25.851380 14549 heartbeater.cc:464] Registering TS with master...
I20260620 09:22:25.851519 14549 heartbeater.cc:511] Master 127.4.59.62:44653 requested a full tablet report, sending...
I20260620 09:22:25.851862 14099 ts_manager.cc:194] Registered new tserver with Master: 911db58da6e645e29284e66861f90c6c (127.4.59.3:43623)
I20260620 09:22:25.861544 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 09:22:25.863554 14138 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:22:25.862902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36598 (local address 127.4.59.62:44653)
0620 09:22:25.863037 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:25.863039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:25.863210 (+ 171us) server_negotiation.cc:407] Connection header received
0620 09:22:25.863292 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:25.863293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:25.863300 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:25.863350 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:25.863483 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36598: BlockingRecv error: recv got EOF from 127.0.0.1:36598 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
I20260620 09:22:25.863605 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 14159
I20260620 09:22:25.867895 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 14290
I20260620 09:22:25.872292 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 14421
I20260620 09:22:25.876406 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 14068
2026-06-20T09:22:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (577 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:22:25 dist-test-slave-414w krb5kdc[14559](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:22:25 dist-test-slave-414w krb5kdc[14559](info): set up 2 sockets
Jun 20 09:22:25 dist-test-slave-414w krb5kdc[14559](info): commencing operation
krb5kdc: starting...
W20260620 09:22:26.909780 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:22:26 dist-test-slave-414w krb5kdc[14559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:22:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:22:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:22:27.027956 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:45913
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:46247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:45913
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:22:27.107297 14572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:27.107540 14572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:27.107579 14572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:27.108726 14572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:22:27.108794 14572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:27.108819 14572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:27.108846 14572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:22:27.108877 14572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:22:27.110316 14572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:46247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:45913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:45913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:27.110574 14572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:27.110792 14572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:27.115481 14577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:27.115666 14572 server_base.cc:1034] running on GCE node
W20260620 09:22:27.115482 14580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:27.115738 14578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:27.115986 14572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:27.116284 14572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:27.117440 14572 hybrid_clock.cc:648] HybridClock initialized: now 1781947347117434 us; error 35 us; skew 500 ppm
Jun 20 09:22:27 dist-test-slave-414w krb5kdc[14559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947347, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:22:27.119836 14572 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:22:27.120282 14572 webserver.cc:460] Webserver started at http://127.4.59.62:42653/ using document root <none> and password file <none>
I20260620 09:22:27.120494 14572 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:27.120568 14572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:27.120716 14572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:27.121753 14572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "aa5d191ce7874a75ba92a6f20876c1c2"
format_stamp: "Formatted at 2026-06-20 09:22:27 on dist-test-slave-414w"
server_key: "dfb26777826049466f785fa166825955"
server_key_iv: "0ba716a2fff81f8d73ffd4bf03568d9d"
server_key_version: "encryptionkey@0"
I20260620 09:22:27.122095 14572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "aa5d191ce7874a75ba92a6f20876c1c2"
format_stamp: "Formatted at 2026-06-20 09:22:27 on dist-test-slave-414w"
server_key: "dfb26777826049466f785fa166825955"
server_key_iv: "0ba716a2fff81f8d73ffd4bf03568d9d"
server_key_version: "encryptionkey@0"
I20260620 09:22:27.123359 14572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:22:27.124078 14587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:27.124272 14572 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:22:27.124373 14572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "aa5d191ce7874a75ba92a6f20876c1c2"
format_stamp: "Formatted at 2026-06-20 09:22:27 on dist-test-slave-414w"
server_key: "dfb26777826049466f785fa166825955"
server_key_iv: "0ba716a2fff81f8d73ffd4bf03568d9d"
server_key_version: "encryptionkey@0"
I20260620 09:22:27.124445 14572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:27.131794 14572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:27.133915 14572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:27.134071 14572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:27.137966 14572 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:45913
I20260620 09:22:27.138024 14639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:45913 every 8 connection(s)
I20260620 09:22:27.138366 14572 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:22:27.139273 14640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:27.141376 14640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2: Bootstrap starting.
I20260620 09:22:27.141889 14640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:27.142045 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 14572
I20260620 09:22:27.142110 14640 log.cc:824] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2: Log is configured to *not* fsync() on all Append() calls
I20260620 09:22:27.142153 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:22:27.142357 4332 external_mini_cluster.cc:1428] Setting key f5984d5da84a636c4552758b4ca8737f
I20260620 09:22:27.142891 14640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2: No bootstrap required, opened a new log
I20260620 09:22:27.145582 14640 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5d191ce7874a75ba92a6f20876c1c2" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45913 } }
I20260620 09:22:27.145745 14640 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:27.145780 14640 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa5d191ce7874a75ba92a6f20876c1c2, State: Initialized, Role: FOLLOWER
I20260620 09:22:27.145900 14640 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5d191ce7874a75ba92a6f20876c1c2" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45913 } }
I20260620 09:22:27.145980 14640 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:22:27.146008 14640 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:22:27.146050 14640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:27.146639 14640 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5d191ce7874a75ba92a6f20876c1c2" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45913 } }
I20260620 09:22:27.146728 14640 leader_election.cc:302] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa5d191ce7874a75ba92a6f20876c1c2; no voters:
I20260620 09:22:27.146903 14640 leader_election.cc:288] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:22:27.146929 14645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:22:27.147087 14645 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [term 1 LEADER]: Becoming Leader. State: Replica: aa5d191ce7874a75ba92a6f20876c1c2, State: Running, Role: LEADER
I20260620 09:22:27.147168 14640 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:22:27.147229 14645 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5d191ce7874a75ba92a6f20876c1c2" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45913 } }
I20260620 09:22:27.148689 14646 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa5d191ce7874a75ba92a6f20876c1c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5d191ce7874a75ba92a6f20876c1c2" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45913 } } }
I20260620 09:22:27.148787 14646 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [sys.catalog]: This master's current role is: LEADER
Jun 20 09:22:27 dist-test-slave-414w krb5kdc[14559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:22:27.149266 14647 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa5d191ce7874a75ba92a6f20876c1c2. Latest consensus state: current_term: 1 leader_uuid: "aa5d191ce7874a75ba92a6f20876c1c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5d191ce7874a75ba92a6f20876c1c2" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45913 } } }
I20260620 09:22:27.149341 14647 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2 [sys.catalog]: This master's current role is: LEADER
I20260620 09:22:27.149354 14653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:22:27.149828 14653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:22:27.151489 14653 catalog_manager.cc:1347] Generated new cluster ID: da2e0715d8bb4f188fa4e3a1b7add1a9
I20260620 09:22:27.151538 14653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:22:27.153196 14644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.142830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43420 (local address 127.4.59.62:45913)
0620 09:22:27.143006 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:27.143013 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:27.143036 (+ 23us) server_negotiation.cc:407] Connection header received
0620 09:22:27.143421 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:27.143427 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:27.143469 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:27.143573 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:27.144310 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.145278 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.146312 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.146716 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.146833 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.149578 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:27.149582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:27.149582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:27.149593 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:27.151534 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:27.151964 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:27.151967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:27.151967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:27.152074 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:27.152284 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:27.152284 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:27.152285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:27.152580 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:27.152741 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:27.152814 (+ 73us) server_negotiation.cc:299] Negotiation successful
0620 09:22:27.152947 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260620 09:22:27.178542 14653 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:22:27.179275 14653 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:22:27.189049 14653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2: Generated new TSK 0
I20260620 09:22:27.189286 14653 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:22:27.224037 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45913
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:46247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:22:27.311797 14667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:27.312089 14667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:27.312134 14667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:27.313375 14667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:27.313417 14667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:27.313443 14667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:22:27.314852 14667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:46247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:27.315095 14667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:27.315663 14667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:27.320822 14672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:27.320797 14675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:27.320811 14673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:27.320962 14667 server_base.cc:1034] running on GCE node
I20260620 09:22:27.321229 14667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:27.321476 14667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:27.322659 14667 hybrid_clock.cc:648] HybridClock initialized: now 1781947347322651 us; error 40 us; skew 500 ppm
Jun 20 09:22:27 dist-test-slave-414w krb5kdc[14559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947347, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:22:27.325304 14667 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:22:27.325726 14667 webserver.cc:460] Webserver started at http://127.4.59.1:42349/ using document root <none> and password file <none>
I20260620 09:22:27.326023 14667 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:27.326110 14667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:27.326243 14667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:27.327402 14667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "658ca2c33c5f46cbbe86b41766209f37"
format_stamp: "Formatted at 2026-06-20 09:22:27 on dist-test-slave-414w"
server_key: "31baef0f7bb754e519787072c58b5dd9"
server_key_iv: "17ba62d961f8ac0e1bc7d8d14e58c609"
server_key_version: "encryptionkey@0"
I20260620 09:22:27.327739 14667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "658ca2c33c5f46cbbe86b41766209f37"
format_stamp: "Formatted at 2026-06-20 09:22:27 on dist-test-slave-414w"
server_key: "31baef0f7bb754e519787072c58b5dd9"
server_key_iv: "17ba62d961f8ac0e1bc7d8d14e58c609"
server_key_version: "encryptionkey@0"
I20260620 09:22:27.329238 14667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260620 09:22:27.330024 14682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:27.330215 14667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:22:27.330313 14667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "658ca2c33c5f46cbbe86b41766209f37"
format_stamp: "Formatted at 2026-06-20 09:22:27 on dist-test-slave-414w"
server_key: "31baef0f7bb754e519787072c58b5dd9"
server_key_iv: "17ba62d961f8ac0e1bc7d8d14e58c609"
server_key_version: "encryptionkey@0"
I20260620 09:22:27.330396 14667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:27.357084 14667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:27.358894 14667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:27.359074 14667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:27.359313 14667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:22:27.359624 14667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:27.359678 14667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:27.359719 14667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:27.359746 14667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:27.365413 14667 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:38781
I20260620 09:22:27.365460 14795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:38781 every 8 connection(s)
I20260620 09:22:27.365799 14667 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:22:27.369364 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 14667
I20260620 09:22:27.369441 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:22:27.369630 4332 external_mini_cluster.cc:1428] Setting key 1b90c525519d7ecf33525a58efa177f3
Jun 20 09:22:27 dist-test-slave-414w krb5kdc[14559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947347, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:22:27.375605 14644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.366534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:57003 (local address 127.4.59.62:45913)
0620 09:22:27.366599 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:27.366601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:27.366781 (+ 180us) server_negotiation.cc:407] Connection header received
0620 09:22:27.367265 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:27.367266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:27.367275 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:27.367331 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:27.368244 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.368750 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.369986 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.370231 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.370305 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.372527 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:27.372529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:27.372530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:27.372544 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:27.374051 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:27.374615 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:27.374616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:27.374617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:27.374679 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:27.374941 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:27.374942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:27.374942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:27.375059 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:27.375143 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:27.375426 (+ 283us) server_negotiation.cc:299] Negotiation successful
0620 09:22:27.375514 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 09:22:27.375793 14798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.366507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45913 (local address 127.4.59.1:57003)
0620 09:22:27.366658 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:27.366698 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:27.367182 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:27.367381 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:27.367384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:27.367926 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:27.368132 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:27.368137 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.368842 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.368843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:27.369852 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:27.369855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.370323 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.370325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:27.370464 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.370972 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:27.370980 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:27.372414 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:27.374214 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:27.374223 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:27.374237 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:27.374518 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:27.374747 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:27.374748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:27.374748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:27.374865 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:27.375135 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:27.375141 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:27.375287 (+ 146us) client_negotiation.cc:769] Sending connection context
0620 09:22:27.375393 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 09:22:27.375555 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260620 09:22:27.376366 14796 heartbeater.cc:347] Connected to a master server at 127.4.59.62:45913
I20260620 09:22:27.376459 14796 heartbeater.cc:464] Registering TS with master...
I20260620 09:22:27.376654 14796 heartbeater.cc:511] Master 127.4.59.62:45913 requested a full tablet report, sending...
I20260620 09:22:27.377276 14604 ts_manager.cc:194] Registered new tserver with Master: 658ca2c33c5f46cbbe86b41766209f37 (127.4.59.1:38781)
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
I20260620 09:22:27.378230 14604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:57003
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:22:27.396343 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45913
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:46247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:22:27.476709 14801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:27.476902 14801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:27.476926 14801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:27.478132 14801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:27.478178 14801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:27.478204 14801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:22:27.479615 14801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:46247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:27.479858 14801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:27.480085 14801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:27.484905 14809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:27.484958 14801 server_base.cc:1034] running on GCE node
W20260620 09:22:27.484961 14807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:27.484920 14806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:27.485375 14801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:27.485633 14801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:27.486824 14801 hybrid_clock.cc:648] HybridClock initialized: now 1781947347486808 us; error 27 us; skew 500 ppm
Jun 20 09:22:27 dist-test-slave-414w krb5kdc[14559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947347, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:22:27.489408 14801 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:22:27.489835 14801 webserver.cc:460] Webserver started at http://127.4.59.2:38279/ using document root <none> and password file <none>
I20260620 09:22:27.490067 14801 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:27.490136 14801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:27.490298 14801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:27.491358 14801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "3c54dcc3aaf0466b9ed86785bc2c0eae"
format_stamp: "Formatted at 2026-06-20 09:22:27 on dist-test-slave-414w"
server_key: "a750bb0a64c2d5d3a66683a8a893b3c8"
server_key_iv: "ff1683a42432f42e3b044a1db5ee05df"
server_key_version: "encryptionkey@0"
I20260620 09:22:27.491737 14801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "3c54dcc3aaf0466b9ed86785bc2c0eae"
format_stamp: "Formatted at 2026-06-20 09:22:27 on dist-test-slave-414w"
server_key: "a750bb0a64c2d5d3a66683a8a893b3c8"
server_key_iv: "ff1683a42432f42e3b044a1db5ee05df"
server_key_version: "encryptionkey@0"
I20260620 09:22:27.493000 14801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:22:27.493829 14816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:27.494038 14801 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:22:27.494140 14801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "3c54dcc3aaf0466b9ed86785bc2c0eae"
format_stamp: "Formatted at 2026-06-20 09:22:27 on dist-test-slave-414w"
server_key: "a750bb0a64c2d5d3a66683a8a893b3c8"
server_key_iv: "ff1683a42432f42e3b044a1db5ee05df"
server_key_version: "encryptionkey@0"
I20260620 09:22:27.494220 14801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:27.532655 14801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:27.534857 14801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:27.535017 14801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:27.535259 14801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:22:27.535564 14801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:27.535619 14801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:27.535663 14801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:27.535687 14801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:27.541867 14801 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:40189
I20260620 09:22:27.541911 14929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:40189 every 8 connection(s)
I20260620 09:22:27.542291 14801 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
Jun 20 09:22:27 dist-test-slave-414w krb5kdc[14559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947347, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:22:27.551489 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 14801
I20260620 09:22:27.551568 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:22:27.551808 4332 external_mini_cluster.cc:1428] Setting key 8d7a91204ee8fff98c4ca98282b999e2
I20260620 09:22:27.551872 14644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.543059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:43183 (local address 127.4.59.62:45913)
0620 09:22:27.543192 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:27.543194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:27.543348 (+ 154us) server_negotiation.cc:407] Connection header received
0620 09:22:27.543880 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:27.543881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:27.543892 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:27.543953 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:27.544881 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.545451 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.546540 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.546738 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.546821 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.549110 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:27.549112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:27.549113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:27.549120 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:27.550415 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:27.550838 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:27.550839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:27.550840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:27.550898 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:27.551155 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:27.551156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:27.551156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:27.551267 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:27.551370 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:27.551683 (+ 313us) server_negotiation.cc:299] Negotiation successful
0620 09:22:27.551769 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260620 09:22:27.551994 14932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.543022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45913 (local address 127.4.59.2:43183)
0620 09:22:27.543207 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:27.543260 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:27.543775 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:27.544032 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:27.544036 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:27.544592 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:27.544778 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:27.544783 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.545559 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.545560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:27.546439 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:27.546441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.546850 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.546852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:27.546981 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.547485 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:27.547494 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:27.549009 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:27.550521 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:27.550527 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:27.550529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:27.550747 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:27.550967 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:27.550968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:27.550968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:27.551077 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:27.551361 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:27.551368 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:27.551507 (+ 139us) client_negotiation.cc:769] Sending connection context
0620 09:22:27.551626 (+ 119us) client_negotiation.cc:240] Negotiation successful
0620 09:22:27.551766 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":80,"threads_started":1}
I20260620 09:22:27.552785 14930 heartbeater.cc:347] Connected to a master server at 127.4.59.62:45913
I20260620 09:22:27.552893 14930 heartbeater.cc:464] Registering TS with master...
I20260620 09:22:27.553081 14930 heartbeater.cc:511] Master 127.4.59.62:45913 requested a full tablet report, sending...
I20260620 09:22:27.553503 14604 ts_manager.cc:194] Registered new tserver with Master: 3c54dcc3aaf0466b9ed86785bc2c0eae (127.4.59.2:40189)
I20260620 09:22:27.553997 14604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:43183
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:22:27.576207 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45913
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:46247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:22:27.656304 14935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:22:27.656502 14935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:22:27.656524 14935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:22:27.657747 14935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:22:27.657786 14935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:22:27.657814 14935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:22:27.659246 14935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:46247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:22:27.659488 14935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:22:27.659718 14935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:22:27.664600 14940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:22:27.664647 14941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:27.664705 14935 server_base.cc:1034] running on GCE node
W20260620 09:22:27.664637 14943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:22:27.665020 14935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:22:27.665330 14935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:22:27.666467 14935 hybrid_clock.cc:648] HybridClock initialized: now 1781947347666458 us; error 30 us; skew 500 ppm
Jun 20 09:22:27 dist-test-slave-414w krb5kdc[14559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947347, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:22:27.668908 14935 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:22:27.669353 14935 webserver.cc:460] Webserver started at http://127.4.59.3:42333/ using document root <none> and password file <none>
I20260620 09:22:27.669545 14935 fs_manager.cc:359] Metadata directory not provided
I20260620 09:22:27.669587 14935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:22:27.669674 14935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:22:27.670655 14935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "483e7c9d70794c378e650444a1cee1f2"
format_stamp: "Formatted at 2026-06-20 09:22:27 on dist-test-slave-414w"
server_key: "519cac7425b0f6795ad139e6e065c37d"
server_key_iv: "01acb016c9742fba9d26e85bf28cc8dc"
server_key_version: "encryptionkey@0"
I20260620 09:22:27.670935 14935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "483e7c9d70794c378e650444a1cee1f2"
format_stamp: "Formatted at 2026-06-20 09:22:27 on dist-test-slave-414w"
server_key: "519cac7425b0f6795ad139e6e065c37d"
server_key_iv: "01acb016c9742fba9d26e85bf28cc8dc"
server_key_version: "encryptionkey@0"
I20260620 09:22:27.672139 14935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:22:27.672830 14950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:27.673000 14935 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:22:27.673116 14935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "483e7c9d70794c378e650444a1cee1f2"
format_stamp: "Formatted at 2026-06-20 09:22:27 on dist-test-slave-414w"
server_key: "519cac7425b0f6795ad139e6e065c37d"
server_key_iv: "01acb016c9742fba9d26e85bf28cc8dc"
server_key_version: "encryptionkey@0"
I20260620 09:22:27.673223 14935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:22:27.691330 14935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:22:27.693620 14935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:22:27.693779 14935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:22:27.694007 14935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:22:27.694309 14935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:22:27.694362 14935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:27.694403 14935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:22:27.694432 14935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:22:27.700124 14935 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:36593
I20260620 09:22:27.700199 15063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:36593 every 8 connection(s)
I20260620 09:22:27.700505 14935 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
Jun 20 09:22:27 dist-test-slave-414w krb5kdc[14559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947347, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:22:27.709260 14644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.701235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:49659 (local address 127.4.59.62:45913)
0620 09:22:27.701304 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:27.701307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:27.701485 (+ 178us) server_negotiation.cc:407] Connection header received
0620 09:22:27.701954 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:27.701955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:27.701965 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:27.702040 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:27.702901 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.703385 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.704335 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.704519 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.704580 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.706875 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:27.706876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:27.706877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:27.706883 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:27.708106 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:27.708453 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:27.708453 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:27.708454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:27.708504 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:27.708702 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:27.708703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:27.708703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:27.708801 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:27.708850 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:27.709060 (+ 210us) server_negotiation.cc:299] Negotiation successful
0620 09:22:27.709118 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 09:22:27.709389 15066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.701220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45913 (local address 127.4.59.3:49659)
0620 09:22:27.701359 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:27.701399 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:27.701872 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:27.702106 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:27.702109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:27.702642 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:22:27.702826 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:27.702829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.703461 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.703461 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:27.704249 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:27.704251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.704597 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.704597 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:27.704692 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.705229 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:22:27.705237 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:22:27.706786 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:22:27.708178 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:27.708184 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:27.708186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:27.708387 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:27.708561 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:22:27.708561 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:22:27.708562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:22:27.708649 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:22:27.708853 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:22:27.708857 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:22:27.708958 (+ 101us) client_negotiation.cc:769] Sending connection context
0620 09:22:27.709034 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 09:22:27.709199 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260620 09:22:27.709857 15064 heartbeater.cc:347] Connected to a master server at 127.4.59.62:45913
I20260620 09:22:27.709962 15064 heartbeater.cc:464] Registering TS with master...
I20260620 09:22:27.710131 15064 heartbeater.cc:511] Master 127.4.59.62:45913 requested a full tablet report, sending...
I20260620 09:22:27.710557 14604 ts_manager.cc:194] Registered new tserver with Master: 483e7c9d70794c378e650444a1cee1f2 (127.4.59.3:36593)
I20260620 09:22:27.710582 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 14935
I20260620 09:22:27.710680 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:22:27.710856 4332 external_mini_cluster.cc:1428] Setting key 7bb6865e0f9adc5370fb13ccca4fe957
I20260620 09:22:27.711004 14604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:49659
I20260620 09:22:27.711725 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:22:27.718904 14644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.713047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43436 (local address 127.4.59.62:45913)
0620 09:22:27.713122 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:27.713124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:27.713246 (+ 122us) server_negotiation.cc:407] Connection header received
0620 09:22:27.713320 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:27.713320 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:27.713330 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:27.713385 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:22:27.713917 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.714378 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.715084 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.715280 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.715364 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.716499 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:22:27.716501 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:22:27.716501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:22:27.716508 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:22:27.717774 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:27.718138 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:27.718139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:27.718139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:27.718195 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:22:27.718388 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:22:27.718389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:22:27.718390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:22:27.718525 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:22:27.718631 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:27.718738 (+ 107us) server_negotiation.cc:299] Negotiation successful
0620 09:22:27.718809 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 09:22:27.720227 14604 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:22:27.721887 14604 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:22:27.728379 15076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.723927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:38781 (local address 127.0.0.1:51410)
0620 09:22:27.724075 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:27.724104 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:27.724156 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:27.724821 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:27.724822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:27.724824 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:22:27.724859 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:27.724861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.726169 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.726170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:27.727100 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:27.727103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.728073 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.728074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:27.728151 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.728164 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 09:22:27.728242 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 09:22:27.728297 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":2,"thread_start_us":79,"threads_started":1}
I20260620 09:22:27.728410 15080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.724484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51410 (local address 127.4.59.1:38781)
0620 09:22:27.724654 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:27.724658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:27.724667 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:22:27.724703 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:27.724705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:27.724721 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:27.724925 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:22:27.724991 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.726056 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.727186 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.727964 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.728064 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.728123 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:27.728260 (+ 137us) server_negotiation.cc:299] Negotiation successful
0620 09:22:27.728302 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
I20260620 09:22:27.728547 15078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.724275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43820 (local address 127.4.59.2:40189)
0620 09:22:27.724446 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:27.724450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:27.724756 (+ 306us) server_negotiation.cc:407] Connection header received
0620 09:22:27.724878 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:27.724881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:27.724897 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:27.724949 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:22:27.725203 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.725875 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.727398 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.728100 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.728160 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.728205 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:27.728401 (+ 196us) server_negotiation.cc:299] Negotiation successful
0620 09:22:27.728446 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260620 09:22:27.728747 15075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.724160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:40189 (local address 127.0.0.1:43820)
0620 09:22:27.724640 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:27.724650 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:27.724784 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:27.725043 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:27.725044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:27.725045 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:22:27.725081 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:27.725083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.725984 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.725985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:27.727287 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:27.727290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.728243 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.728244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:27.728303 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.728308 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 09:22:27.728639 (+ 331us) client_negotiation.cc:240] Negotiation successful
0620 09:22:27.728684 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":45,"thread_start_us":64,"threads_started":1}
I20260620 09:22:27.728844 15077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.723927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:36593 (local address 127.0.0.1:42724)
0620 09:22:27.724264 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:27.724280 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:27.724330 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:27.724626 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:27.724629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:27.724632 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:22:27.724693 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:27.724695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.726013 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.726014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:27.727538 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:27.727539 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.728350 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.728351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:27.728412 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.728416 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:22:27.728731 (+ 315us) client_negotiation.cc:240] Negotiation successful
0620 09:22:27.728774 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":16,"thread_start_us":54,"threads_started":1}
I20260620 09:22:27.729161 14730 tablet_service.cc:1467] Processing CreateTablet for tablet 39460517d7004f32937fdf24ba5a9c4d (DEFAULT_TABLE table=test-table [id=c9d7a83823b84c9abb7319dcdf3bc923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:22:27.729326 15079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.724291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42724 (local address 127.4.59.3:36593)
0620 09:22:27.724446 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:27.724450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:27.724460 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:22:27.724495 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:27.724497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:27.724513 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:27.724596 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:22:27.724792 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.725893 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.727614 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.728262 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.729095 (+ 833us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.729185 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:27.729210 (+ 25us) server_negotiation.cc:299] Negotiation successful
0620 09:22:27.729248 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260620 09:22:27.729519 14730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39460517d7004f32937fdf24ba5a9c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:27.729488 14864 tablet_service.cc:1467] Processing CreateTablet for tablet 39460517d7004f32937fdf24ba5a9c4d (DEFAULT_TABLE table=test-table [id=c9d7a83823b84c9abb7319dcdf3bc923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:22:27.729792 14864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39460517d7004f32937fdf24ba5a9c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:27.729905 14998 tablet_service.cc:1467] Processing CreateTablet for tablet 39460517d7004f32937fdf24ba5a9c4d (DEFAULT_TABLE table=test-table [id=c9d7a83823b84c9abb7319dcdf3bc923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:22:27.730180 14998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39460517d7004f32937fdf24ba5a9c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:22:27.731822 15081 tablet_bootstrap.cc:492] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37: Bootstrap starting.
I20260620 09:22:27.732348 15081 tablet_bootstrap.cc:654] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:27.732969 15081 log.cc:824] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37: Log is configured to *not* fsync() on all Append() calls
I20260620 09:22:27.733311 15082 tablet_bootstrap.cc:492] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae: Bootstrap starting.
I20260620 09:22:27.733733 15081 tablet_bootstrap.cc:492] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37: No bootstrap required, opened a new log
I20260620 09:22:27.733803 15081 ts_tablet_manager.cc:1397] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260620 09:22:27.733902 15082 tablet_bootstrap.cc:654] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:27.734310 15082 log.cc:824] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae: Log is configured to *not* fsync() on all Append() calls
I20260620 09:22:27.734874 15082 tablet_bootstrap.cc:492] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae: No bootstrap required, opened a new log
I20260620 09:22:27.734922 15082 ts_tablet_manager.cc:1397] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:22:27.735400 15081 raft_consensus.cc:348] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658ca2c33c5f46cbbe86b41766209f37" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38781 } } peers { permanent_uuid: "3c54dcc3aaf0466b9ed86785bc2c0eae" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40189 } } peers { permanent_uuid: "483e7c9d70794c378e650444a1cee1f2" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36593 } }
I20260620 09:22:27.735574 15081 raft_consensus.cc:374] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:27.735622 15081 raft_consensus.cc:729] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658ca2c33c5f46cbbe86b41766209f37, State: Initialized, Role: FOLLOWER
I20260620 09:22:27.735673 15084 tablet_bootstrap.cc:492] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2: Bootstrap starting.
I20260620 09:22:27.735756 15081 consensus_queue.cc:260] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658ca2c33c5f46cbbe86b41766209f37" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38781 } } peers { permanent_uuid: "3c54dcc3aaf0466b9ed86785bc2c0eae" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40189 } } peers { permanent_uuid: "483e7c9d70794c378e650444a1cee1f2" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36593 } }
I20260620 09:22:27.736150 14796 heartbeater.cc:503] Master 127.4.59.62:45913 was elected leader, sending a full tablet report...
I20260620 09:22:27.736268 15084 tablet_bootstrap.cc:654] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2: Neither blocks nor log segments found. Creating new log.
I20260620 09:22:27.736132 15081 ts_tablet_manager.cc:1428] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 09:22:27.736819 15084 log.cc:824] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2: Log is configured to *not* fsync() on all Append() calls
I20260620 09:22:27.736717 15082 raft_consensus.cc:348] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658ca2c33c5f46cbbe86b41766209f37" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38781 } } peers { permanent_uuid: "3c54dcc3aaf0466b9ed86785bc2c0eae" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40189 } } peers { permanent_uuid: "483e7c9d70794c378e650444a1cee1f2" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36593 } }
I20260620 09:22:27.736881 15082 raft_consensus.cc:374] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:27.736959 15082 raft_consensus.cc:729] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c54dcc3aaf0466b9ed86785bc2c0eae, State: Initialized, Role: FOLLOWER
I20260620 09:22:27.737126 15082 consensus_queue.cc:260] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658ca2c33c5f46cbbe86b41766209f37" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38781 } } peers { permanent_uuid: "3c54dcc3aaf0466b9ed86785bc2c0eae" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40189 } } peers { permanent_uuid: "483e7c9d70794c378e650444a1cee1f2" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36593 } }
I20260620 09:22:27.737404 15082 ts_tablet_manager.cc:1428] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 09:22:27.737438 14930 heartbeater.cc:503] Master 127.4.59.62:45913 was elected leader, sending a full tablet report...
I20260620 09:22:27.737617 15084 tablet_bootstrap.cc:492] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2: No bootstrap required, opened a new log
I20260620 09:22:27.737664 15084 ts_tablet_manager.cc:1397] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:22:27.739135 15084 raft_consensus.cc:348] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658ca2c33c5f46cbbe86b41766209f37" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38781 } } peers { permanent_uuid: "3c54dcc3aaf0466b9ed86785bc2c0eae" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40189 } } peers { permanent_uuid: "483e7c9d70794c378e650444a1cee1f2" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36593 } }
I20260620 09:22:27.739291 15084 raft_consensus.cc:374] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:22:27.739338 15084 raft_consensus.cc:729] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 483e7c9d70794c378e650444a1cee1f2, State: Initialized, Role: FOLLOWER
I20260620 09:22:27.739451 15084 consensus_queue.cc:260] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658ca2c33c5f46cbbe86b41766209f37" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38781 } } peers { permanent_uuid: "3c54dcc3aaf0466b9ed86785bc2c0eae" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40189 } } peers { permanent_uuid: "483e7c9d70794c378e650444a1cee1f2" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36593 } }
I20260620 09:22:27.739637 15084 ts_tablet_manager.cc:1428] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:22:27.739691 15064 heartbeater.cc:503] Master 127.4.59.62:45913 was elected leader, sending a full tablet report...
W20260620 09:22:27.792842 14931 tablet.cc:2369] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 09:22:27.866532 14797 tablet.cc:2369] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:22:27.890447 15086 raft_consensus.cc:482] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:22:27.890604 15086 raft_consensus.cc:504] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658ca2c33c5f46cbbe86b41766209f37" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38781 } } peers { permanent_uuid: "3c54dcc3aaf0466b9ed86785bc2c0eae" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40189 } } peers { permanent_uuid: "483e7c9d70794c378e650444a1cee1f2" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36593 } }
I20260620 09:22:27.890986 15086 leader_election.cc:288] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c54dcc3aaf0466b9ed86785bc2c0eae (127.4.59.2:40189), 483e7c9d70794c378e650444a1cee1f2 (127.4.59.3:36593)
I20260620 09:22:27.893960 15079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.891324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:39191 (local address 127.4.59.3:36593)
0620 09:22:27.891371 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:27.891373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:27.891457 (+ 84us) server_negotiation.cc:407] Connection header received
0620 09:22:27.891528 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:27.891529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:27.891552 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:27.891602 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:22:27.891769 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.892245 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.893221 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.893607 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.893662 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.893692 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:27.893841 (+ 149us) server_negotiation.cc:299] Negotiation successful
0620 09:22:27.893887 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 09:22:27.893985 15078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.891270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:44581 (local address 127.4.59.2:40189)
0620 09:22:27.891345 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:27.891347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:27.891504 (+ 157us) server_negotiation.cc:407] Connection header received
0620 09:22:27.891547 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:27.891548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:27.891566 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:27.891634 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:22:27.891815 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.892257 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.893259 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.893616 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.893703 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.893712 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:27.893882 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 09:22:27.893924 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 09:22:27.893990 15091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.891197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:36593 (local address 127.4.59.1:39191)
0620 09:22:27.891374 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:27.891385 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:27.891448 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:27.891623 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:27.891624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:27.891627 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:22:27.891704 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:27.891706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.892321 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.892322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:27.893102 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:27.893104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.893685 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.893687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:27.893767 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.893771 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:22:27.893817 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 09:22:27.893872 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":93,"threads_started":1}
I20260620 09:22:27.894004 15090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.891198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:40189 (local address 127.4.59.1:44581)
0620 09:22:27.891390 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:27.891400 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:22:27.891492 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:22:27.891649 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:22:27.891650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:22:27.891653 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:22:27.891719 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:27.891721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.892354 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.892355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:27.893112 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:22:27.893114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.893716 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.893717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:22:27.893787 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.893790 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:22:27.893865 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 09:22:27.893901 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":87,"threads_started":1}
I20260620 09:22:27.894357 15018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39460517d7004f32937fdf24ba5a9c4d" candidate_uuid: "658ca2c33c5f46cbbe86b41766209f37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "483e7c9d70794c378e650444a1cee1f2" is_pre_election: true
I20260620 09:22:27.894516 15018 raft_consensus.cc:2454] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 658ca2c33c5f46cbbe86b41766209f37 in term 0.
I20260620 09:22:27.894778 14685 leader_election.cc:302] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 483e7c9d70794c378e650444a1cee1f2, 658ca2c33c5f46cbbe86b41766209f37; no voters:
I20260620 09:22:27.894909 14884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39460517d7004f32937fdf24ba5a9c4d" candidate_uuid: "658ca2c33c5f46cbbe86b41766209f37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c54dcc3aaf0466b9ed86785bc2c0eae" is_pre_election: true
I20260620 09:22:27.895061 14884 raft_consensus.cc:2454] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 658ca2c33c5f46cbbe86b41766209f37 in term 0.
I20260620 09:22:27.895187 15086 raft_consensus.cc:2781] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:22:27.895255 15086 raft_consensus.cc:482] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:22:27.895293 15086 raft_consensus.cc:3037] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:27.895975 15086 raft_consensus.cc:504] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658ca2c33c5f46cbbe86b41766209f37" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38781 } } peers { permanent_uuid: "3c54dcc3aaf0466b9ed86785bc2c0eae" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40189 } } peers { permanent_uuid: "483e7c9d70794c378e650444a1cee1f2" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36593 } }
I20260620 09:22:27.896142 15086 leader_election.cc:288] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [CANDIDATE]: Term 1 election: Requested vote from peers 3c54dcc3aaf0466b9ed86785bc2c0eae (127.4.59.2:40189), 483e7c9d70794c378e650444a1cee1f2 (127.4.59.3:36593)
I20260620 09:22:27.896342 15018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39460517d7004f32937fdf24ba5a9c4d" candidate_uuid: "658ca2c33c5f46cbbe86b41766209f37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "483e7c9d70794c378e650444a1cee1f2"
I20260620 09:22:27.896425 15018 raft_consensus.cc:3037] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:27.896441 14884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39460517d7004f32937fdf24ba5a9c4d" candidate_uuid: "658ca2c33c5f46cbbe86b41766209f37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c54dcc3aaf0466b9ed86785bc2c0eae"
I20260620 09:22:27.896535 14884 raft_consensus.cc:3037] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:22:27.897184 15018 raft_consensus.cc:2454] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658ca2c33c5f46cbbe86b41766209f37 in term 1.
I20260620 09:22:27.897343 14685 leader_election.cc:302] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 483e7c9d70794c378e650444a1cee1f2, 658ca2c33c5f46cbbe86b41766209f37; no voters:
I20260620 09:22:27.897396 14884 raft_consensus.cc:2454] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658ca2c33c5f46cbbe86b41766209f37 in term 1.
I20260620 09:22:27.897455 15086 raft_consensus.cc:2781] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:22:27.897641 15086 raft_consensus.cc:686] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [term 1 LEADER]: Becoming Leader. State: Replica: 658ca2c33c5f46cbbe86b41766209f37, State: Running, Role: LEADER
I20260620 09:22:27.897740 15086 consensus_queue.cc:237] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658ca2c33c5f46cbbe86b41766209f37" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38781 } } peers { permanent_uuid: "3c54dcc3aaf0466b9ed86785bc2c0eae" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40189 } } peers { permanent_uuid: "483e7c9d70794c378e650444a1cee1f2" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36593 } }
I20260620 09:22:27.898556 14604 catalog_manager.cc:5495] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 658ca2c33c5f46cbbe86b41766209f37 (127.4.59.1). New cstate: current_term: 1 leader_uuid: "658ca2c33c5f46cbbe86b41766209f37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658ca2c33c5f46cbbe86b41766209f37" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c54dcc3aaf0466b9ed86785bc2c0eae" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "483e7c9d70794c378e650444a1cee1f2" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36593 } health_report { overall_health: UNKNOWN } } }
I20260620 09:22:27.911536 15080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.909220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51422 (local address 127.4.59.1:38781)
0620 09:22:27.909273 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 09:22:27.909275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:22:27.909284 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:22:27.909331 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:22:27.909335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:22:27.909355 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:22:27.909429 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:22:27.909559 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.910014 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.910771 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:22:27.910962 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:22:27.911038 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:22:27.911210 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:22:27.911304 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:22:27.911398 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 09:22:27.911416 (+ 18us) server_negotiation.cc:299] Negotiation successful
0620 09:22:27.911437 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 09:22:27.913380 15018 raft_consensus.cc:1261] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2 [term 1 FOLLOWER]: Refusing update from remote peer 658ca2c33c5f46cbbe86b41766209f37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:22:27.913372 14884 raft_consensus.cc:1261] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae [term 1 FOLLOWER]: Refusing update from remote peer 658ca2c33c5f46cbbe86b41766209f37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:22:27.913717 15086 consensus_queue.cc:1035] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "483e7c9d70794c378e650444a1cee1f2" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:22:27.913885 15092 consensus_queue.cc:1035] T 39460517d7004f32937fdf24ba5a9c4d P 658ca2c33c5f46cbbe86b41766209f37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c54dcc3aaf0466b9ed86785bc2c0eae" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:22:27.916635 15096 mvcc.cc:205] Tried to move back new op lower bound from 7298856337050501120 to 7298856336989700096. Current Snapshot: MvccSnapshot[applied={T|T < 7298856337050501120 or (T in {7298856337050501120})}]
I20260620 09:22:27.918973 14604 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:22:27.919066 14604 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:22:27.919857 14604 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa5d191ce7874a75ba92a6f20876c1c2: Sending DeleteTablet for 3 replicas of tablet 39460517d7004f32937fdf24ba5a9c4d
I20260620 09:22:27.920172 14864 tablet_service.cc:1514] Processing DeleteTablet for tablet 39460517d7004f32937fdf24ba5a9c4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:22:27 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:43820
I20260620 09:22:27.920238 14998 tablet_service.cc:1514] Processing DeleteTablet for tablet 39460517d7004f32937fdf24ba5a9c4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:22:27 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:42724
I20260620 09:22:27.920410 15104 tablet_replica.cc:330] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae: stopping tablet replica
I20260620 09:22:27.920423 14730 tablet_service.cc:1514] Processing DeleteTablet for tablet 39460517d7004f32937fdf24ba5a9c4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:22:27 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:51410
I20260620 09:22:27.920502 15104 raft_consensus.cc:2229] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:22:27.920594 15104 raft_consensus.cc:2258] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:22:27.920585 14729 tablet_service.cc:1514] Processing DeleteTablet for tablet 39460517d7004f32937fdf24ba5a9c4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:22:27 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:51410
I20260620 09:22:27.920604 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 14667
I20260620 09:22:27.921485 15104 ts_tablet_manager.cc:1905] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:22:27.923432 15104 ts_tablet_manager.cc:1918] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:22:27.923509 15104 log.cc:1196] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/39460517d7004f32937fdf24ba5a9c4d
I20260620 09:22:27.923787 15104 ts_tablet_manager.cc:1939] T 39460517d7004f32937fdf24ba5a9c4d P 3c54dcc3aaf0466b9ed86785bc2c0eae: Deleting consensus metadata
I20260620 09:22:27.923974 15105 tablet_replica.cc:330] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2: stopping tablet replica
I20260620 09:22:27.924084 15105 raft_consensus.cc:2229] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:22:27.924185 15105 raft_consensus.cc:2258] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:22:27.924577 15105 ts_tablet_manager.cc:1905] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:22:27.924611 14588 catalog_manager.cc:4841] TS 3c54dcc3aaf0466b9ed86785bc2c0eae (127.4.59.2:40189): tablet 39460517d7004f32937fdf24ba5a9c4d (table test-table [id=c9d7a83823b84c9abb7319dcdf3bc923]) successfully deleted
I20260620 09:22:27.926436 15105 ts_tablet_manager.cc:1918] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:22:27.926513 15105 log.cc:1196] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/39460517d7004f32937fdf24ba5a9c4d
I20260620 09:22:27.926779 15105 ts_tablet_manager.cc:1939] T 39460517d7004f32937fdf24ba5a9c4d P 483e7c9d70794c378e650444a1cee1f2: Deleting consensus metadata
I20260620 09:22:27.927196 14590 catalog_manager.cc:4841] TS 483e7c9d70794c378e650444a1cee1f2 (127.4.59.3:36593): tablet 39460517d7004f32937fdf24ba5a9c4d (table test-table [id=c9d7a83823b84c9abb7319dcdf3bc923]) successfully deleted
W20260620 09:22:27.928227 14591 connection.cc:535] server connection from 127.4.59.1:57003 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 09:22:27.928256 14588 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.59.1:38781 (error 108)
I20260620 09:22:27.928766 15077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:22:27.928629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:38781 (local address 127.0.0.1:51436)
0620 09:22:27.928695 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0620 09:22:27.928736 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:38781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 09:22:27.928771 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 14801
W20260620 09:22:27.931233 14588 catalog_manager.cc:4584] TS 658ca2c33c5f46cbbe86b41766209f37 (127.4.59.1:38781): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 39460517d7004f32937fdf24ba5a9c4d: Network error: Client connection negotiation failed: client connection to 127.4.59.1:38781: connect: Connection refused (error 111)
I20260620 09:22:27.934361 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 14935
I20260620 09:22:27.938800 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 14572
2026-06-20T09:22:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2063 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:22:27 dist-test-slave-414w krb5kdc[15110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:22:27 dist-test-slave-414w krb5kdc[15110](info): set up 2 sockets
Jun 20 09:22:27 dist-test-slave-414w krb5kdc[15110](info): commencing operation
krb5kdc: starting...
W20260620 09:22:28.973655 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:22:29 dist-test-slave-414w krb5kdc[15110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:22:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:22:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.59.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.59.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.59.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/rangeradmin_127.4.59.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.59.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/rangeradmin_127.4.59.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.59.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.59.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.59.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/rangerlookup_127.4.59.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.59.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/rangerlookup_127.4.59.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.59.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.59.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.59.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/HTTP_127.4.59.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.59.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/HTTP_127.4.59.20@KRBTEST.COM.keytab.
I20260620 09:22:29.134158 4332 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task0Fx1M2/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/postgres -l logfile start
2026-06-20 09:22:31.655 UTC [15140] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 09:22:31.655 UTC [15140] LOG: listening on IPv4 address "127.4.59.20", port 36603
2026-06-20 09:22:31.659 UTC [15140] LOG: listening on Unix socket "/tmp/.s.PGSQL.36603"
2026-06-20 09:22:31.694 UTC [15143] LOG: database system was shut down at 2026-06-20 09:22:30 UTC
2026-06-20 09:22:31.699 UTC [15140] LOG: database system is ready to accept connections
I20260620 09:22:32.641666 4332 mini_postgres.cc:96] Postgres bound to 36603
2026-06-20 09:22:32.647 UTC [15151] FATAL: database "slave" does not exist
127.4.59.20:36603 - accepting connections
I20260620 09:22:32.648505 4332 mini_ranger.cc:161] Starting Ranger...
I20260620 09:22:32.663031 4332 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 09:22:33.145340 4332 mini_ranger.cc:87] Created ranger Postgres database
I20260620 09:22:33.145422 4332 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-admin
2026-06-20 09:22:33,408 [I] DB FLAVOR :POSTGRES
2026-06-20 09:22:33,409 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 09:22:33,409 [I] Checking connection..
2026-06-20 09:22:33,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 09:22:33,667 [I] Checking connection passed.
2026-06-20 09:22:33,668 [I] --------- Verifying version history table ---------
2026-06-20 09:22:33,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 09:22:33,902 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 09:22:33,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 09:22:34,136 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 09:22:34,137 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 09:22:34,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 09:22:34.360 UTC [15215] WARNING: there is no transaction in progress
2026-06-20 09:22:34,366 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 09:22:34,367 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 09:22:34,606 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 09:22:34,607 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 09:22:34,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 09:22:34,856 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 09:22:35,111 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-414w.c.gcp-upstream.internal','N') ;"
2026-06-20 09:22:35,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 09:22:35,573 [I] Table x_portal_user does not exist in database ranger
2026-06-20 09:22:35,573 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 09:22:35,844 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 09:22:35,844 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 09:22:36,077 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 09:22:36,300 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 09:22:36,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 09:22:37.295 UTC [15323] WARNING: there is no transaction in progress
2026-06-20 09:22:37.312 UTC [15323] WARNING: there is no transaction in progress
2026-06-20 09:22:37.330 UTC [15323] WARNING: there is no transaction in progress
2026-06-20 09:22:37.345 UTC [15323] WARNING: there is no transaction in progress
2026-06-20 09:22:37.361 UTC [15323] WARNING: there is no transaction in progress
2026-06-20 09:22:37.381 UTC [15323] WARNING: there is no transaction in progress
2026-06-20 09:22:37.448 UTC [15323] WARNING: there is no transaction in progress
2026-06-20 09:22:37.455 UTC [15323] WARNING: there is no transaction in progress
2026-06-20 09:22:37.471 UTC [15323] WARNING: there is no transaction in progress
2026-06-20 09:22:37.482 UTC [15323] WARNING: there is no transaction in progress
2026-06-20 09:22:37.493 UTC [15323] WARNING: there is no transaction in progress
2026-06-20 09:22:37.507 UTC [15323] WARNING: there is no transaction in progress
2026-06-20 09:22:37.518 UTC [15323] WARNING: there is no transaction in progress
2026-06-20 09:22:37.524 UTC [15323] WARNING: there is no transaction in progress
2026-06-20 09:22:37.531 UTC [15323] WARNING: there is no transaction in progress
2026-06-20 09:22:37.538 UTC [15323] WARNING: there is no transaction in progress
2026-06-20 09:22:38,177 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 09:22:38,177 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 09:22:38,409 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 09:22:38,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 09:22:38,655 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 09:22:38,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 09:22:38,911 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 09:22:38,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 09:22:39,145 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 09:22:39,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-414w.c.gcp-upstream.internal';"
2026-06-20 09:22:39,632 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 09:22:39,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 09:22:39,891 [I] DB_PATCHES have already been applied
I20260620 09:22:39.896727 4332 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-admin:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task0Fx1M2/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 09:22:39.896824 4332 mini_ranger.cc:193] Using host: 127.4.59.20
I20260620 09:22:39.899675 4332 mini_ranger.cc:227] Ranger admin URL: http://127.4.59.20:50213
Jun 20, 2026 9:22:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 9:22:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 9:22:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 9:22:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 9:22:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 9:22:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 9:22:40 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 09:22:41.028558 4332 mini_ranger.cc:160] Time spent starting Ranger: real 8.380s user 0.002s sys 0.006s
Jun 20, 2026 9:23:00 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 9:23:00 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 20, 2026 9:23:00 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 9:23:01 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 9:23:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 9:23:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 9:23:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 9:23:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 9:23:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 9:23:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 9:23:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 9:23:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 9:23:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 9:23:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 9:23:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 9:23:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 9:23:01 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 9:23:01 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 09:23:02.518903 4332 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.59.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.59.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.59.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/rangerkms_127.4.59.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.59.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/rangerkms_127.4.59.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.59.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.59.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.59.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/HTTP_127.4.59.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.59.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/HTTP_127.4.59.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 09:23:02 dist-test-slave-414w krb5kdc[15110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947382, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 09:23:02.603626 4332 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 09:23:02.613077 4332 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 09:23:03.563064 4332 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 09:23:03.563149 4332 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-kms
I20260620 09:23:03.563170 4332 mini_ranger_kms.cc:227] Using postgres at 127.4.59.20:36603
2026-06-20 09:23:03,730 [I] DB FLAVOR :POSTGRES
2026-06-20 09:23:03,731 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 09:23:03,731 [I] Checking connection
2026-06-20 09:23:03,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 09:23:03,993 [I] connection success
2026-06-20 09:23:03,993 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 09:23:03,993 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 09:23:03,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 09:23:04,227 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 09:23:04,227 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 09:23:04,227 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 09:23:04,228 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:36603/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 09:23:04,510 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 09:23:04.976934 4332 mini_ranger_kms.cc:310] Created kms service
I20260620 09:23:05.174196 4332 mini_ranger_kms.cc:326] Created kudu user
I20260620 09:23:05.218719 4332 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 09:23:05 dist-test-slave-414w krb5kdc[15110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947382, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
I20260620 09:23:05.631198 4332 mini_ranger_kms.cc:384] Added ranger policy
I20260620 09:23:05.631310 4332 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-kms:/tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 09:23:05.631332 4332 mini_ranger_kms.cc:240] Using host: 127.4.59.20
I20260620 09:23:05.635444 4332 mini_ranger_kms.cc:276] Ranger KMS PID: 15503
I20260620 09:23:05.635571 4332 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.59.20:53779
Jun 20, 2026 9:23:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 9:23:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 9:23:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 9:23:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 9:23:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 9:23:07 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260620 09:23:07.990343 4332 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.387s user 0.003s sys 0.149s
I20260620 09:23:07.990464 4332 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 09:23:07.990586 4332 mini_ranger_kms.cc:401] 127.4.59.20:53779/kms/v1/keys
Jun 20 09:23:08 dist-test-slave-414w krb5kdc[15110](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781947388, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.59.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 20 09:23:08 dist-test-slave-414w krb5kdc[15110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947388, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.59.20@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
Jun 20 09:23:08 dist-test-slave-414w krb5kdc[15110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947388, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.59.20@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
Jun 20 09:23:10 dist-test-slave-414w krb5kdc[15110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:23:10.933877 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:37385
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:37397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:37385
--ranger_config_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.59.20:53779/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:23:11.016069 15554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:23:11.016261 15554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:23:11.016281 15554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:23:11.017473 15554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:23:11.017510 15554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:23:11.017524 15554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:23:11.017536 15554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:23:11.017549 15554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:23:11.019088 15554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:37397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.59.20:53779/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:37385
--ranger_config_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:37385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:23:11.019320 15554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:23:11.019554 15554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:23:11.025676 15559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:11.025722 15562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:11.025782 15560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:23:11.025842 15554 server_base.cc:1034] running on GCE node
I20260620 09:23:11.026078 15554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:23:11.026398 15554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:23:11.027580 15554 hybrid_clock.cc:648] HybridClock initialized: now 1781947391027568 us; error 40 us; skew 500 ppm
Jun 20 09:23:11 dist-test-slave-414w krb5kdc[15110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947391, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:23:11.033754 15554 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:23:11.034250 15554 webserver.cc:460] Webserver started at http://127.4.59.62:36519/ using document root <none> and password file <none>
I20260620 09:23:11.034490 15554 fs_manager.cc:359] Metadata directory not provided
I20260620 09:23:11.034529 15554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:23:11.034655 15554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 09:23:11 dist-test-slave-414w krb5kdc[15110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947391, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
I20260620 09:23:11.077689 15554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "1cd4e439f1314bafbe30479b52e95089"
format_stamp: "Formatted at 2026-06-20 09:23:11 on dist-test-slave-414w"
server_key: "0e709b04011867eb3ff43b7d116d7222"
server_key_iv: "5db9e8ba89d8325448fe619e6417c202"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:11.078109 15554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "1cd4e439f1314bafbe30479b52e95089"
format_stamp: "Formatted at 2026-06-20 09:23:11 on dist-test-slave-414w"
server_key: "0e709b04011867eb3ff43b7d116d7222"
server_key_iv: "5db9e8ba89d8325448fe619e6417c202"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:11.079627 15554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:23:11.090528 15569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:11.090734 15554 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 09:23:11.090833 15554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "1cd4e439f1314bafbe30479b52e95089"
format_stamp: "Formatted at 2026-06-20 09:23:11 on dist-test-slave-414w"
server_key: "0e709b04011867eb3ff43b7d116d7222"
server_key_iv: "5db9e8ba89d8325448fe619e6417c202"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:11.090940 15554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:23:11.102814 15554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:23:11.105044 15554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:23:11.105229 15554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:23:11.109979 15554 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:37385
I20260620 09:23:11.110002 15621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:37385 every 8 connection(s)
I20260620 09:23:11.110340 15554 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:23:11.110988 15622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:23:11.113211 15622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089: Bootstrap starting.
I20260620 09:23:11.113816 15622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089: Neither blocks nor log segments found. Creating new log.
I20260620 09:23:11.114090 15622 log.cc:824] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089: Log is configured to *not* fsync() on all Append() calls
I20260620 09:23:11.114750 15622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089: No bootstrap required, opened a new log
I20260620 09:23:11.117103 15622 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd4e439f1314bafbe30479b52e95089" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37385 } }
I20260620 09:23:11.117312 15622 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:23:11.117341 15622 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cd4e439f1314bafbe30479b52e95089, State: Initialized, Role: FOLLOWER
I20260620 09:23:11.117506 15622 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd4e439f1314bafbe30479b52e95089" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37385 } }
I20260620 09:23:11.117574 15622 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:23:11.117623 15622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:23:11.117668 15622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:23:11.118204 15622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd4e439f1314bafbe30479b52e95089" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37385 } }
I20260620 09:23:11.118285 15622 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cd4e439f1314bafbe30479b52e95089; no voters:
I20260620 09:23:11.118494 15622 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:23:11.118511 15625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:23:11.118625 15625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [term 1 LEADER]: Becoming Leader. State: Replica: 1cd4e439f1314bafbe30479b52e95089, State: Running, Role: LEADER
I20260620 09:23:11.118705 15625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd4e439f1314bafbe30479b52e95089" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37385 } }
I20260620 09:23:11.118747 15622 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:23:11.119007 15625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cd4e439f1314bafbe30479b52e95089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd4e439f1314bafbe30479b52e95089" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37385 } } }
I20260620 09:23:11.119151 15625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [sys.catalog]: This master's current role is: LEADER
I20260620 09:23:11.119194 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 15554
I20260620 09:23:11.119308 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:23:11.119351 15625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cd4e439f1314bafbe30479b52e95089. Latest consensus state: current_term: 1 leader_uuid: "1cd4e439f1314bafbe30479b52e95089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd4e439f1314bafbe30479b52e95089" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37385 } } }
I20260620 09:23:11.119412 15625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089 [sys.catalog]: This master's current role is: LEADER
I20260620 09:23:11.119488 4332 external_mini_cluster.cc:1428] Setting key 245ab12e2b324dc115de11573b475808
I20260620 09:23:11.119534 15632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:23:11.120003 15632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:23:11.120265 15622 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 09:23:11.129701 15632 catalog_manager.cc:1347] Generated new cluster ID: 74e448d83a0d46e68bd1df6194f84c6b
I20260620 09:23:11.129768 15632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 09:23:11 dist-test-slave-414w krb5kdc[15110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:23:11.144703 15632 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:23:11.145491 15632 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:23:11.146688 15642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:11.119988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56078 (local address 127.4.59.62:37385)
0620 09:23:11.120446 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:11.120455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:11.120485 (+ 30us) server_negotiation.cc:407] Connection header received
0620 09:23:11.120600 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:11.120607 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:11.120661 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:11.120785 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:23:11.122635 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:11.124617 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:11.126026 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:11.126864 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:11.126979 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:11.135445 (+ 8466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:23:11.135450 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:23:11.135450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:23:11.135462 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:23:11.137521 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:11.145505 (+ 7984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:11.145508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:11.145508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:11.145615 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:11.145809 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:11.145810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:11.145811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:11.146088 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:23:11.146255 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:11.146331 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 09:23:11.146469 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":73,"threads_started":1}
I20260620 09:23:11.156694 15632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089: Generated new TSK 0
I20260620 09:23:11.156917 15632 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:23:11.908013 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.59.20:53779/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:37385
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:37397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:23:12.028913 15667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:23:12.029225 15667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:23:12.029259 15667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:23:12.030901 15667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:23:12.030972 15667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:23:12.031013 15667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:23:12.033120 15667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:37397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.59.20:53779/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:37385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:23:12.033387 15667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:23:12.033633 15667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:23:12.039173 15672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:12.039297 15675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:23:12.039191 15667 server_base.cc:1034] running on GCE node
W20260620 09:23:12.039187 15673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:23:12.039633 15667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:23:12.039877 15667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:23:12.041033 15667 hybrid_clock.cc:648] HybridClock initialized: now 1781947392041011 us; error 41 us; skew 500 ppm
Jun 20 09:23:12 dist-test-slave-414w krb5kdc[15110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947392, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:23:12.043648 15667 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:23:12.044137 15667 webserver.cc:460] Webserver started at http://127.4.59.1:34415/ using document root <none> and password file <none>
I20260620 09:23:12.044410 15667 fs_manager.cc:359] Metadata directory not provided
I20260620 09:23:12.044472 15667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:23:12.044564 15667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 09:23:12 dist-test-slave-414w krb5kdc[15110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947392, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
I20260620 09:23:12.060956 15667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "ba0610d7a08d4243a4c77378d489fa80"
format_stamp: "Formatted at 2026-06-20 09:23:12 on dist-test-slave-414w"
server_key: "85a8b8e35766dda7d1e2402882e25865"
server_key_iv: "efd148f6f2a55ddc13cf781dd349dd64"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:12.061570 15667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "ba0610d7a08d4243a4c77378d489fa80"
format_stamp: "Formatted at 2026-06-20 09:23:12 on dist-test-slave-414w"
server_key: "85a8b8e35766dda7d1e2402882e25865"
server_key_iv: "efd148f6f2a55ddc13cf781dd349dd64"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:12.064806 15667 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260620 09:23:12.078492 15682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:12.078714 15667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:23:12.078816 15667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "ba0610d7a08d4243a4c77378d489fa80"
format_stamp: "Formatted at 2026-06-20 09:23:12 on dist-test-slave-414w"
server_key: "85a8b8e35766dda7d1e2402882e25865"
server_key_iv: "efd148f6f2a55ddc13cf781dd349dd64"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:12.078918 15667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:23:12.105412 15667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:23:12.107895 15667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:23:12.108057 15667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:23:12.108314 15667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:23:12.108690 15667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:23:12.108733 15667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:12.108763 15667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:23:12.108809 15667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:12.115100 15667 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:37197
I20260620 09:23:12.115578 15667 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:23:12.115795 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 15667
I20260620 09:23:12.115890 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:23:12.116070 4332 external_mini_cluster.cc:1428] Setting key af8292c97d4cf78dfbc86a02a8c8724f
Jun 20 09:23:12 dist-test-slave-414w krb5kdc[15110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947392, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:23:12.128072 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.117017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:37385 (local address 127.4.59.1:35735)
0620 09:23:12.117568 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:12.117622 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:12.118224 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:12.118640 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:12.118644 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:12.118897 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:23:12.119184 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:12.119191 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.120178 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.120179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:12.121508 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:12.121511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.122012 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.122013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:12.122153 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.122728 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:23:12.122738 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:23:12.124376 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:23:12.126507 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:12.126513 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:12.126515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:12.126793 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:12.127011 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:12.127011 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:12.127012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:12.127135 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:12.127415 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:23:12.127419 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:23:12.127603 (+ 184us) client_negotiation.cc:769] Sending connection context
0620 09:23:12.127690 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 09:23:12.127839 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":76,"threads_started":1}
I20260620 09:23:12.128705 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.116998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:35735 (local address 127.4.59.62:37385)
0620 09:23:12.118419 (+ 1421us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:12.118422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:12.118430 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:23:12.118454 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:12.118455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:12.118466 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:12.119279 (+ 813us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:23:12.119332 (+ 53us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.120056 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.121608 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.121916 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.122386 (+ 470us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.124491 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:23:12.124492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:23:12.124493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:23:12.124499 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:23:12.126400 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:12.126878 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:12.126879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:12.126879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:12.126938 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:12.127211 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:12.127211 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:12.127212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:12.127342 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:23:12.128486 (+ 1144us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:12.128528 (+ 42us) server_negotiation.cc:299] Negotiation successful
0620 09:23:12.128609 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1392,"thread_start_us":83,"threads_started":1}
I20260620 09:23:12.129132 15798 heartbeater.cc:347] Connected to a master server at 127.4.59.62:37385
I20260620 09:23:12.129220 15797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:37197 every 8 connection(s)
I20260620 09:23:12.129249 15798 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:12.129423 15798 heartbeater.cc:511] Master 127.4.59.62:37385 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
I20260620 09:23:12.129978 15586 ts_manager.cc:194] Registered new tserver with Master: ba0610d7a08d4243a4c77378d489fa80 (127.4.59.1:37197)
I20260620 09:23:12.130853 15586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:35735
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:23:12.149636 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.59.20:53779/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:37385
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:37397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:23:12.255589 15804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:23:12.255797 15804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:23:12.255836 15804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:23:12.257041 15804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:23:12.257105 15804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:23:12.257151 15804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:23:12.258674 15804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:37397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.59.20:53779/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:37385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:23:12.258920 15804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:23:12.259150 15804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:23:12.264045 15812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:12.264055 15814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:12.264271 15811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:23:12.264509 15804 server_base.cc:1034] running on GCE node
I20260620 09:23:12.264740 15804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:23:12.265010 15804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:23:12.266173 15804 hybrid_clock.cc:648] HybridClock initialized: now 1781947392266161 us; error 27 us; skew 500 ppm
Jun 20 09:23:12 dist-test-slave-414w krb5kdc[15110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947392, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:23:12.268783 15804 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:23:12.269248 15804 webserver.cc:460] Webserver started at http://127.4.59.2:44535/ using document root <none> and password file <none>
I20260620 09:23:12.269474 15804 fs_manager.cc:359] Metadata directory not provided
I20260620 09:23:12.269541 15804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:23:12.269671 15804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 09:23:12 dist-test-slave-414w krb5kdc[15110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947392, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
I20260620 09:23:12.281672 15804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "01c6973bd98b4c3d8b94aaafe1424fb5"
format_stamp: "Formatted at 2026-06-20 09:23:12 on dist-test-slave-414w"
server_key: "f58c3f07957a29e4ead9a141ff8c420f"
server_key_iv: "7ef532cf311c72e117a1901e597369a3"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:12.282166 15804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "01c6973bd98b4c3d8b94aaafe1424fb5"
format_stamp: "Formatted at 2026-06-20 09:23:12 on dist-test-slave-414w"
server_key: "f58c3f07957a29e4ead9a141ff8c420f"
server_key_iv: "7ef532cf311c72e117a1901e597369a3"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:12.284302 15804 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
Jun 20 09:23:12 dist-test-slave-414w krb5kdc[15110](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781947392, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:23:12.312610 15821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:12.312916 15804 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.000s
I20260620 09:23:12.313033 15804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "01c6973bd98b4c3d8b94aaafe1424fb5"
format_stamp: "Formatted at 2026-06-20 09:23:12 on dist-test-slave-414w"
server_key: "f58c3f07957a29e4ead9a141ff8c420f"
server_key_iv: "7ef532cf311c72e117a1901e597369a3"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:12.313158 15804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:23:12.325997 15804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:23:12.328297 15804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:23:12.328501 15804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:23:12.328778 15804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:23:12.329190 15804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:23:12.329280 15804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:12.329357 15804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:23:12.329420 15804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:12.336046 15804 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:42825
I20260620 09:23:12.336318 15934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:42825 every 8 connection(s)
I20260620 09:23:12.336591 15804 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:23:12.340163 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 15804
I20260620 09:23:12.340265 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:23:12.340456 4332 external_mini_cluster.cc:1428] Setting key dfa6152dbf5003cec0f38b6bd5a66825
Jun 20 09:23:12 dist-test-slave-414w krb5kdc[15110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947392, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:23:12.347465 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.337286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:41683 (local address 127.4.59.62:37385)
0620 09:23:12.337343 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:12.337346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:12.337915 (+ 569us) server_negotiation.cc:407] Connection header received
0620 09:23:12.338599 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:12.338600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:12.338611 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:12.338679 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:23:12.339323 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.340125 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.341581 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.341892 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.341930 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.344351 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:23:12.344353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:23:12.344353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:23:12.344361 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:23:12.345967 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:12.346402 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:12.346403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:12.346403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:12.346470 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:12.346745 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:12.346746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:12.346747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:12.346892 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:23:12.346972 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:12.347275 (+ 303us) server_negotiation.cc:299] Negotiation successful
0620 09:23:12.347369 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 09:23:12.347633 15937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.337386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:37385 (local address 127.4.59.2:41683)
0620 09:23:12.337759 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:12.337807 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:12.338493 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:12.338750 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:12.338753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:12.338974 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:23:12.339235 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:12.339241 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.340222 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.340223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:12.341486 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:12.341489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.342008 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.342009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:12.342136 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.342731 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:23:12.342740 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:23:12.344241 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:23:12.346060 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:12.346065 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:12.346068 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:12.346325 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:12.346552 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:12.346553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:12.346553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:12.346669 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:12.346967 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:23:12.346972 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:23:12.347124 (+ 152us) client_negotiation.cc:769] Sending connection context
0620 09:23:12.347227 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 09:23:12.347369 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":83,"threads_started":1}
I20260620 09:23:12.348145 15935 heartbeater.cc:347] Connected to a master server at 127.4.59.62:37385
I20260620 09:23:12.348246 15935 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:12.348433 15935 heartbeater.cc:511] Master 127.4.59.62:37385 requested a full tablet report, sending...
I20260620 09:23:12.348838 15586 ts_manager.cc:194] Registered new tserver with Master: 01c6973bd98b4c3d8b94aaafe1424fb5 (127.4.59.2:42825)
I20260620 09:23:12.349349 15586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:41683
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:23:12.372213 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.59.20:53779/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:37385
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:37397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:23:12.494022 15941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:23:12.494351 15941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:23:12.494442 15941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:23:12.496124 15941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:23:12.496305 15941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:23:12.496402 15941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:23:12.498585 15941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:37397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.59.20:53779/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:37385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:23:12.498986 15941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:23:12.499289 15941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:23:12.505854 15947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:12.506207 15949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:12.507098 15946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:23:12.507225 15941 server_base.cc:1034] running on GCE node
I20260620 09:23:12.507436 15941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:23:12.507704 15941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:23:12.508879 15941 hybrid_clock.cc:648] HybridClock initialized: now 1781947392508856 us; error 35 us; skew 500 ppm
Jun 20 09:23:12 dist-test-slave-414w krb5kdc[15110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947392, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:23:12.512183 15941 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:23:12.512710 15941 webserver.cc:460] Webserver started at http://127.4.59.3:33311/ using document root <none> and password file <none>
I20260620 09:23:12.512962 15941 fs_manager.cc:359] Metadata directory not provided
I20260620 09:23:12.513016 15941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:23:12.513168 15941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 09:23:12 dist-test-slave-414w krb5kdc[15110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947392, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
I20260620 09:23:12.527968 15941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "040fc088dba64b0d8815936505d4de4a"
format_stamp: "Formatted at 2026-06-20 09:23:12 on dist-test-slave-414w"
server_key: "1db9398c5c757c9c8abc15a4a0c0020b"
server_key_iv: "665ddfef16bfc241e6a011170e26a504"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:12.528533 15941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "040fc088dba64b0d8815936505d4de4a"
format_stamp: "Formatted at 2026-06-20 09:23:12 on dist-test-slave-414w"
server_key: "1db9398c5c757c9c8abc15a4a0c0020b"
server_key_iv: "665ddfef16bfc241e6a011170e26a504"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:12.542855 15941 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.001s sys 0.000s
I20260620 09:23:12.557284 15956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:12.557525 15941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 09:23:12.557623 15941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "040fc088dba64b0d8815936505d4de4a"
format_stamp: "Formatted at 2026-06-20 09:23:12 on dist-test-slave-414w"
server_key: "1db9398c5c757c9c8abc15a4a0c0020b"
server_key_iv: "665ddfef16bfc241e6a011170e26a504"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:12.557716 15941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:23:12.582357 15941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:23:12.584767 15941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:23:12.584941 15941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:23:12.585234 15941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:23:12.585649 15941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:23:12.585734 15941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:12.585804 15941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:23:12.585860 15941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:12.594645 15941 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:42695
I20260620 09:23:12.595027 15941 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:23:12.600140 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 15941
I20260620 09:23:12.600240 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:23:12.600406 4332 external_mini_cluster.cc:1428] Setting key 379313a6765f56b6a0963f8e8aea2821
I20260620 09:23:12.601457 16069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:42695 every 8 connection(s)
Jun 20 09:23:12 dist-test-slave-414w krb5kdc[15110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947392, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:23:12.606664 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.595994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:40951 (local address 127.4.59.62:37385)
0620 09:23:12.596033 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:12.596035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:12.596067 (+ 32us) server_negotiation.cc:407] Connection header received
0620 09:23:12.596765 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:12.596766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:12.596777 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:12.596843 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:23:12.597573 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.598314 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.599827 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.600102 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.600174 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.603465 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:23:12.603467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:23:12.603467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:23:12.603475 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:23:12.605082 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:12.605571 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:12.605572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:12.605572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:12.605635 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:12.605933 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:12.605934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:12.605934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:12.606064 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:23:12.606127 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:12.606487 (+ 360us) server_negotiation.cc:299] Negotiation successful
0620 09:23:12.606569 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:23:12.606859 16072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.595733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:37385 (local address 127.4.59.3:40951)
0620 09:23:12.595927 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:12.595976 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:12.596647 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:12.596917 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:12.596921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:12.597205 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:23:12.597479 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:12.597485 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.598419 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.598420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:12.599721 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:12.599724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.600245 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.600246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:12.600391 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.601064 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:23:12.601074 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:23:12.603350 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:23:12.605192 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:12.605200 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:12.605202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:12.605485 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:12.605713 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:12.605714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:12.605714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:12.605850 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:12.606135 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:23:12.606141 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:23:12.606309 (+ 168us) client_negotiation.cc:769] Sending connection context
0620 09:23:12.606424 (+ 115us) client_negotiation.cc:240] Negotiation successful
0620 09:23:12.606573 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260620 09:23:12.607409 16070 heartbeater.cc:347] Connected to a master server at 127.4.59.62:37385
I20260620 09:23:12.607508 16070 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:12.607678 16070 heartbeater.cc:511] Master 127.4.59.62:37385 requested a full tablet report, sending...
I20260620 09:23:12.608035 15586 ts_manager.cc:194] Registered new tserver with Master: 040fc088dba64b0d8815936505d4de4a (127.4.59.3:42695)
I20260620 09:23:12.608492 15586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:40951
I20260620 09:23:12.612206 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:23:12.633759 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.614149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56090 (local address 127.4.59.62:37385)
0620 09:23:12.614191 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:12.614193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:12.614421 (+ 228us) server_negotiation.cc:407] Connection header received
0620 09:23:12.614520 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:12.614521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:12.614531 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:12.614558 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:23:12.615263 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.615991 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.625203 (+ 9212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.625490 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.625554 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.630709 (+ 5155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:23:12.630711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:23:12.630711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:23:12.630719 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:23:12.632473 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:12.632871 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:12.632872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:12.632873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:12.632938 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:12.633191 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:12.633192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:12.633192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:12.633329 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:23:12.633444 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:12.633583 (+ 139us) server_negotiation.cc:299] Negotiation successful
0620 09:23:12.633661 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 09:23:12.635623 15586 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56090:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:23:12.638125 15586 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:23:12.650629 16085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.641531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:42695 (local address 127.0.0.1:39626)
0620 09:23:12.642841 (+ 1310us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:12.642849 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:12.642883 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:12.643494 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:12.643498 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:12.643499 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:23:12.643575 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:12.643577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.645303 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.645303 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:12.648900 (+ 3597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:12.648904 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.650307 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.650308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:12.650400 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.650414 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 09:23:12.650487 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 09:23:12.650544 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1296,"thread_start_us":57,"threads_started":1}
I20260620 09:23:12.651970 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.642361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46606 (local address 127.4.59.1:37197)
0620 09:23:12.643330 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:12.643334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:12.643498 (+ 164us) server_negotiation.cc:407] Connection header received
0620 09:23:12.643527 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:12.643529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:12.643542 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:12.643630 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:23:12.643809 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.644914 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.650532 (+ 5618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.651320 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.651382 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.651432 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:12.651854 (+ 422us) server_negotiation.cc:299] Negotiation successful
0620 09:23:12.651897 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":941,"thread_start_us":51,"threads_started":1}
I20260620 09:23:12.652287 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.641667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:37197 (local address 127.0.0.1:46606)
0620 09:23:12.643384 (+ 1717us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:12.643393 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:12.643442 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:12.643703 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:12.643704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:12.643706 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:23:12.643738 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:12.643739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.645021 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.645022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:12.646333 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:12.646335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.651495 (+ 5160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.651496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:12.651556 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.651559 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:23:12.651622 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 09:23:12.651661 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1701,"thread_start_us":51,"threads_started":1}
I20260620 09:23:12.652556 16086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.642006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42050 (local address 127.4.59.2:42825)
0620 09:23:12.642988 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:12.642993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:12.643003 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:23:12.643040 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:12.643042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:12.643056 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:12.643135 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:23:12.643751 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.646984 (+ 3233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.651221 (+ 4237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.651947 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.652012 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.652100 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:12.652436 (+ 336us) server_negotiation.cc:299] Negotiation successful
0620 09:23:12.652476 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":948,"thread_start_us":94,"threads_started":1}
I20260620 09:23:12.652877 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.642192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39626 (local address 127.4.59.3:42695)
0620 09:23:12.643258 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:12.643262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:12.643271 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:23:12.643309 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:12.643311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:12.643327 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:12.643408 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:23:12.644172 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.645222 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.649316 (+ 4094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.650114 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.650188 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.650270 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:12.652747 (+ 2477us) server_negotiation.cc:299] Negotiation successful
0620 09:23:12.652791 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1033,"thread_start_us":83,"threads_started":1}
I20260620 09:23:12.653303 15732 tablet_service.cc:1467] Processing CreateTablet for tablet d037e0189c93440c828e38b5bc6f5184 (DEFAULT_TABLE table=test-table [id=6beb92ed96934a1ea13138f218fc58f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:23:12.653680 15732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d037e0189c93440c828e38b5bc6f5184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:23:12.653750 15993 tablet_service.cc:1467] Processing CreateTablet for tablet d037e0189c93440c828e38b5bc6f5184 (DEFAULT_TABLE table=test-table [id=6beb92ed96934a1ea13138f218fc58f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:23:12.654052 15993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d037e0189c93440c828e38b5bc6f5184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:23:12.652549 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.641315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:42825 (local address 127.0.0.1:42050)
0620 09:23:12.642628 (+ 1313us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:12.642671 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:12.642738 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:12.643648 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:12.643649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:12.643650 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:23:12.643677 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:12.643679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.647095 (+ 3416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.647096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:12.649205 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:12.649208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.652133 (+ 2925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.652134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:12.652192 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.652196 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:23:12.652248 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 09:23:12.652286 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1282,"thread_start_us":72,"threads_started":1}
I20260620 09:23:12.657650 16089 tablet_bootstrap.cc:492] T d037e0189c93440c828e38b5bc6f5184 P ba0610d7a08d4243a4c77378d489fa80: Bootstrap starting.
I20260620 09:23:12.658157 15869 tablet_service.cc:1467] Processing CreateTablet for tablet d037e0189c93440c828e38b5bc6f5184 (DEFAULT_TABLE table=test-table [id=6beb92ed96934a1ea13138f218fc58f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:23:12.658488 15869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d037e0189c93440c828e38b5bc6f5184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:23:12.661068 16090 tablet_bootstrap.cc:492] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a: Bootstrap starting.
I20260620 09:23:12.662536 16091 tablet_bootstrap.cc:492] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5: Bootstrap starting.
I20260620 09:23:12.662875 16090 tablet_bootstrap.cc:654] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a: Neither blocks nor log segments found. Creating new log.
I20260620 09:23:12.663528 16091 tablet_bootstrap.cc:654] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5: Neither blocks nor log segments found. Creating new log.
I20260620 09:23:12.663909 16089 tablet_bootstrap.cc:654] T d037e0189c93440c828e38b5bc6f5184 P ba0610d7a08d4243a4c77378d489fa80: Neither blocks nor log segments found. Creating new log.
I20260620 09:23:12.663885 16091 log.cc:824] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5: Log is configured to *not* fsync() on all Append() calls
I20260620 09:23:12.664258 16089 log.cc:824] T d037e0189c93440c828e38b5bc6f5184 P ba0610d7a08d4243a4c77378d489fa80: Log is configured to *not* fsync() on all Append() calls
I20260620 09:23:12.665477 16090 log.cc:824] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a: Log is configured to *not* fsync() on all Append() calls
I20260620 09:23:12.665961 16089 tablet_bootstrap.cc:492] T d037e0189c93440c828e38b5bc6f5184 P ba0610d7a08d4243a4c77378d489fa80: No bootstrap required, opened a new log
I20260620 09:23:12.666786 16089 ts_tablet_manager.cc:1397] T d037e0189c93440c828e38b5bc6f5184 P ba0610d7a08d4243a4c77378d489fa80: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260620 09:23:12.668967 16089 raft_consensus.cc:348] T d037e0189c93440c828e38b5bc6f5184 P ba0610d7a08d4243a4c77378d489fa80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c6973bd98b4c3d8b94aaafe1424fb5" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42825 } } peers { permanent_uuid: "040fc088dba64b0d8815936505d4de4a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 42695 } } peers { permanent_uuid: "ba0610d7a08d4243a4c77378d489fa80" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 37197 } }
I20260620 09:23:12.669520 16090 tablet_bootstrap.cc:492] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a: No bootstrap required, opened a new log
I20260620 09:23:12.669610 16090 ts_tablet_manager.cc:1397] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260620 09:23:12.670941 16090 raft_consensus.cc:348] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c6973bd98b4c3d8b94aaafe1424fb5" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42825 } } peers { permanent_uuid: "040fc088dba64b0d8815936505d4de4a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 42695 } } peers { permanent_uuid: "ba0610d7a08d4243a4c77378d489fa80" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 37197 } }
I20260620 09:23:12.671082 16090 raft_consensus.cc:374] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:23:12.673202 16090 raft_consensus.cc:729] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 040fc088dba64b0d8815936505d4de4a, State: Initialized, Role: FOLLOWER
I20260620 09:23:12.673453 16090 consensus_queue.cc:260] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c6973bd98b4c3d8b94aaafe1424fb5" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42825 } } peers { permanent_uuid: "040fc088dba64b0d8815936505d4de4a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 42695 } } peers { permanent_uuid: "ba0610d7a08d4243a4c77378d489fa80" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 37197 } }
I20260620 09:23:12.673837 16090 ts_tablet_manager.cc:1428] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260620 09:23:12.674005 16089 raft_consensus.cc:374] T d037e0189c93440c828e38b5bc6f5184 P ba0610d7a08d4243a4c77378d489fa80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:23:12.674069 16089 raft_consensus.cc:729] T d037e0189c93440c828e38b5bc6f5184 P ba0610d7a08d4243a4c77378d489fa80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0610d7a08d4243a4c77378d489fa80, State: Initialized, Role: FOLLOWER
I20260620 09:23:12.674189 16089 consensus_queue.cc:260] T d037e0189c93440c828e38b5bc6f5184 P ba0610d7a08d4243a4c77378d489fa80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c6973bd98b4c3d8b94aaafe1424fb5" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42825 } } peers { permanent_uuid: "040fc088dba64b0d8815936505d4de4a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 42695 } } peers { permanent_uuid: "ba0610d7a08d4243a4c77378d489fa80" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 37197 } }
I20260620 09:23:12.674467 15798 heartbeater.cc:503] Master 127.4.59.62:37385 was elected leader, sending a full tablet report...
I20260620 09:23:12.675400 16070 heartbeater.cc:503] Master 127.4.59.62:37385 was elected leader, sending a full tablet report...
I20260620 09:23:12.675431 16089 ts_tablet_manager.cc:1428] T d037e0189c93440c828e38b5bc6f5184 P ba0610d7a08d4243a4c77378d489fa80: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260620 09:23:12.677937 16091 tablet_bootstrap.cc:492] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5: No bootstrap required, opened a new log
I20260620 09:23:12.678011 16091 ts_tablet_manager.cc:1397] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20260620 09:23:12.680033 16091 raft_consensus.cc:348] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c6973bd98b4c3d8b94aaafe1424fb5" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42825 } } peers { permanent_uuid: "040fc088dba64b0d8815936505d4de4a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 42695 } } peers { permanent_uuid: "ba0610d7a08d4243a4c77378d489fa80" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 37197 } }
I20260620 09:23:12.680195 16091 raft_consensus.cc:374] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:23:12.680231 16091 raft_consensus.cc:729] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01c6973bd98b4c3d8b94aaafe1424fb5, State: Initialized, Role: FOLLOWER
I20260620 09:23:12.680337 16091 consensus_queue.cc:260] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c6973bd98b4c3d8b94aaafe1424fb5" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42825 } } peers { permanent_uuid: "040fc088dba64b0d8815936505d4de4a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 42695 } } peers { permanent_uuid: "ba0610d7a08d4243a4c77378d489fa80" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 37197 } }
I20260620 09:23:12.680591 16091 ts_tablet_manager.cc:1428] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:23:12.680846 15935 heartbeater.cc:503] Master 127.4.59.62:37385 was elected leader, sending a full tablet report...
I20260620 09:23:12.685695 16095 raft_consensus.cc:482] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:23:12.685815 16095 raft_consensus.cc:504] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c6973bd98b4c3d8b94aaafe1424fb5" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42825 } } peers { permanent_uuid: "040fc088dba64b0d8815936505d4de4a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 42695 } } peers { permanent_uuid: "ba0610d7a08d4243a4c77378d489fa80" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 37197 } }
I20260620 09:23:12.686158 16095 leader_election.cc:288] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01c6973bd98b4c3d8b94aaafe1424fb5 (127.4.59.2:42825), ba0610d7a08d4243a4c77378d489fa80 (127.4.59.1:37197)
I20260620 09:23:12.690032 16072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.686293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:42825 (local address 127.4.59.3:35041)
0620 09:23:12.686331 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:12.686343 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:12.686385 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:12.686755 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:12.686757 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:12.686759 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:23:12.686821 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:12.686822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.687781 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.687782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:12.688993 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:12.688996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.689792 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.689793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:12.689870 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.689874 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:23:12.689924 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 09:23:12.689962 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260620 09:23:12.690587 16086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.686597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:35041 (local address 127.4.59.2:42825)
0620 09:23:12.686637 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:12.686639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:12.686647 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:23:12.686668 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:12.686668 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:12.686676 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:12.686736 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:23:12.686896 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.687666 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.689092 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.689677 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.689749 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.689760 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:12.690485 (+ 725us) server_negotiation.cc:299] Negotiation successful
0620 09:23:12.690526 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 09:23:12.690902 15884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d037e0189c93440c828e38b5bc6f5184" candidate_uuid: "040fc088dba64b0d8815936505d4de4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01c6973bd98b4c3d8b94aaafe1424fb5" is_pre_election: true
I20260620 09:23:12.691023 15884 raft_consensus.cc:2454] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 040fc088dba64b0d8815936505d4de4a in term 0.
I20260620 09:23:12.691289 15957 leader_election.cc:302] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01c6973bd98b4c3d8b94aaafe1424fb5, 040fc088dba64b0d8815936505d4de4a; no voters:
I20260620 09:23:12.691395 16095 raft_consensus.cc:2781] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:23:12.691429 16095 raft_consensus.cc:482] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:23:12.691447 16095 raft_consensus.cc:3037] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:23:12.692346 16095 raft_consensus.cc:504] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c6973bd98b4c3d8b94aaafe1424fb5" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42825 } } peers { permanent_uuid: "040fc088dba64b0d8815936505d4de4a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 42695 } } peers { permanent_uuid: "ba0610d7a08d4243a4c77378d489fa80" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 37197 } }
I20260620 09:23:12.692463 16095 leader_election.cc:288] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [CANDIDATE]: Term 1 election: Requested vote from peers 01c6973bd98b4c3d8b94aaafe1424fb5 (127.4.59.2:42825), ba0610d7a08d4243a4c77378d489fa80 (127.4.59.1:37197)
I20260620 09:23:12.692736 15884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d037e0189c93440c828e38b5bc6f5184" candidate_uuid: "040fc088dba64b0d8815936505d4de4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01c6973bd98b4c3d8b94aaafe1424fb5"
I20260620 09:23:12.692795 15884 raft_consensus.cc:3037] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:23:12.693655 15884 raft_consensus.cc:2454] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 040fc088dba64b0d8815936505d4de4a in term 1.
I20260620 09:23:12.693895 15957 leader_election.cc:302] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01c6973bd98b4c3d8b94aaafe1424fb5, 040fc088dba64b0d8815936505d4de4a; no voters:
I20260620 09:23:12.694051 16095 raft_consensus.cc:2781] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:23:12.694211 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.686624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:54085 (local address 127.4.59.1:37197)
0620 09:23:12.686790 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:12.686792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:12.690276 (+ 3484us) server_negotiation.cc:407] Connection header received
0620 09:23:12.690302 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:12.690303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:12.690312 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:12.690369 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:23:12.690854 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.691554 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.693034 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.693651 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.693723 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.693735 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:12.694081 (+ 346us) server_negotiation.cc:299] Negotiation successful
0620 09:23:12.694126 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260620 09:23:12.694226 16095 raft_consensus.cc:686] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [term 1 LEADER]: Becoming Leader. State: Replica: 040fc088dba64b0d8815936505d4de4a, State: Running, Role: LEADER
I20260620 09:23:12.694309 16095 consensus_queue.cc:237] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c6973bd98b4c3d8b94aaafe1424fb5" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42825 } } peers { permanent_uuid: "040fc088dba64b0d8815936505d4de4a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 42695 } } peers { permanent_uuid: "ba0610d7a08d4243a4c77378d489fa80" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 37197 } }
I20260620 09:23:12.695086 15585 catalog_manager.cc:5495] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 040fc088dba64b0d8815936505d4de4a (127.4.59.3). New cstate: current_term: 1 leader_uuid: "040fc088dba64b0d8815936505d4de4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c6973bd98b4c3d8b94aaafe1424fb5" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "040fc088dba64b0d8815936505d4de4a" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 42695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0610d7a08d4243a4c77378d489fa80" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 37197 } health_report { overall_health: UNKNOWN } } }
I20260620 09:23:12.695636 16072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.686403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:37197 (local address 127.4.59.3:54085)
0620 09:23:12.690096 (+ 3693us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:12.690107 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:12.690146 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:12.690741 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:12.690742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:12.690744 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:23:12.690773 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:12.690774 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.691915 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.691916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:12.692941 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:12.692944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.693782 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.693783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:12.693863 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.693866 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:23:12.693973 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 09:23:12.694010 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3683,"thread_start_us":94,"threads_started":1}
I20260620 09:23:12.696250 15752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d037e0189c93440c828e38b5bc6f5184" candidate_uuid: "040fc088dba64b0d8815936505d4de4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0610d7a08d4243a4c77378d489fa80" is_pre_election: true
I20260620 09:23:12.696408 15752 raft_consensus.cc:2454] T d037e0189c93440c828e38b5bc6f5184 P ba0610d7a08d4243a4c77378d489fa80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 040fc088dba64b0d8815936505d4de4a in term 0.
I20260620 09:23:12.696511 15751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d037e0189c93440c828e38b5bc6f5184" candidate_uuid: "040fc088dba64b0d8815936505d4de4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0610d7a08d4243a4c77378d489fa80"
I20260620 09:23:12.696565 15751 raft_consensus.cc:3037] T d037e0189c93440c828e38b5bc6f5184 P ba0610d7a08d4243a4c77378d489fa80 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:23:12.697396 15751 raft_consensus.cc:2454] T d037e0189c93440c828e38b5bc6f5184 P ba0610d7a08d4243a4c77378d489fa80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 040fc088dba64b0d8815936505d4de4a in term 1.
I20260620 09:23:12.709174 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.705783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39642 (local address 127.4.59.3:42695)
0620 09:23:12.705822 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:12.705824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:12.705833 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:23:12.705914 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:12.705915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:12.705934 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:12.706005 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:23:12.706201 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.706956 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.708059 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:12.708354 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:12.708434 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:12.708622 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:23:12.708743 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:23:12.708810 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:12.709030 (+ 220us) server_negotiation.cc:299] Negotiation successful
0620 09:23:12.709051 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 09:23:12.711239 15751 raft_consensus.cc:1261] T d037e0189c93440c828e38b5bc6f5184 P ba0610d7a08d4243a4c77378d489fa80 [term 1 FOLLOWER]: Refusing update from remote peer 040fc088dba64b0d8815936505d4de4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:23:12.711627 16099 consensus_queue.cc:1035] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0610d7a08d4243a4c77378d489fa80" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 37197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:23:12.713749 15884 raft_consensus.cc:1261] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5 [term 1 FOLLOWER]: Refusing update from remote peer 040fc088dba64b0d8815936505d4de4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:23:12.714103 16099 consensus_queue.cc:1035] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "01c6973bd98b4c3d8b94aaafe1424fb5" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:23:12.718395 16102 mvcc.cc:205] Tried to move back new op lower bound from 7298856520542392320 to 7298856520476430336. Current Snapshot: MvccSnapshot[applied={T|T < 7298856520542392320}]
I20260620 09:23:12.718865 16105 mvcc.cc:205] Tried to move back new op lower bound from 7298856520542392320 to 7298856520476430336. Current Snapshot: MvccSnapshot[applied={T|T < 7298856520542392320}]
I20260620 09:23:12.720850 16103 mvcc.cc:205] Tried to move back new op lower bound from 7298856520542392320 to 7298856520476430336. Current Snapshot: MvccSnapshot[applied={T|T < 7298856520542392320}]
I20260620 09:23:12.723119 15585 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:23:12.723201 15585 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:23:12.723928 15585 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1cd4e439f1314bafbe30479b52e95089: Sending DeleteTablet for 3 replicas of tablet d037e0189c93440c828e38b5bc6f5184
I20260620 09:23:12.724387 15869 tablet_service.cc:1514] Processing DeleteTablet for tablet d037e0189c93440c828e38b5bc6f5184 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:23:12 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:42050
I20260620 09:23:12.724581 15732 tablet_service.cc:1514] Processing DeleteTablet for tablet d037e0189c93440c828e38b5bc6f5184 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:23:12 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:46606
I20260620 09:23:12.724933 15993 tablet_service.cc:1514] Processing DeleteTablet for tablet d037e0189c93440c828e38b5bc6f5184 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:23:12 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:39626
I20260620 09:23:12.725512 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 15667
I20260620 09:23:12.727702 16111 tablet_replica.cc:330] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5: stopping tablet replica
I20260620 09:23:12.727849 16113 tablet_replica.cc:330] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a: stopping tablet replica
I20260620 09:23:12.728015 16113 raft_consensus.cc:2229] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:23:12.728142 16111 raft_consensus.cc:2229] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:23:12.728284 16111 raft_consensus.cc:2258] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:23:12.728451 16113 raft_consensus.cc:2258] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:23:12.729009 16111 ts_tablet_manager.cc:1905] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:23:12.729470 16113 ts_tablet_manager.cc:1905] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:23:12.734462 16113 ts_tablet_manager.cc:1918] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:23:12.741201 16113 log.cc:1196] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/d037e0189c93440c828e38b5bc6f5184
I20260620 09:23:12.741400 16111 ts_tablet_manager.cc:1918] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:23:12.743498 16113 ts_tablet_manager.cc:1939] T d037e0189c93440c828e38b5bc6f5184 P 040fc088dba64b0d8815936505d4de4a: Deleting consensus metadata
I20260620 09:23:12.743625 16111 log.cc:1196] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/d037e0189c93440c828e38b5bc6f5184
I20260620 09:23:12.744271 16111 ts_tablet_manager.cc:1939] T d037e0189c93440c828e38b5bc6f5184 P 01c6973bd98b4c3d8b94aaafe1424fb5: Deleting consensus metadata
I20260620 09:23:12.744764 15570 catalog_manager.cc:4841] TS 040fc088dba64b0d8815936505d4de4a (127.4.59.3:42695): tablet d037e0189c93440c828e38b5bc6f5184 (table test-table [id=6beb92ed96934a1ea13138f218fc58f6]) successfully deleted
I20260620 09:23:12.745332 15570 catalog_manager.cc:4841] TS 01c6973bd98b4c3d8b94aaafe1424fb5 (127.4.59.2:42825): tablet d037e0189c93440c828e38b5bc6f5184 (table test-table [id=6beb92ed96934a1ea13138f218fc58f6]) successfully deleted
W20260620 09:23:12.745574 15570 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.59.1:37197 (error 108)
I20260620 09:23:12.752120 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 15804
I20260620 09:23:12.758075 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 15941
I20260620 09:23:12.759372 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:12.759254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:37197 (local address 127.0.0.1:46620)
0620 09:23:12.759303 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:12.759336 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:37197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 09:23:12.759582 15570 catalog_manager.cc:4584] TS ba0610d7a08d4243a4c77378d489fa80 (127.4.59.1:37197): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d037e0189c93440c828e38b5bc6f5184: Network error: Client connection negotiation failed: client connection to 127.4.59.1:37197: connect: Connection refused (error 111)
I20260620 09:23:12.763765 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 15554
I20260620 09:23:12.768363 4332 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 09:23:13.112551 4332 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 09:23:13.112661 4332 mini_ranger.cc:66] Stopping Ranger...
I20260620 09:23:13.456809 4332 mini_ranger.cc:68] Stopped Ranger
2026-06-20 09:23:13.457 UTC [15140] LOG: received smart shutdown request
2026-06-20 09:23:13.460 UTC [15140] LOG: background worker "logical replication launcher" (PID 15149) exited with exit code 1
2026-06-20 09:23:13.460 UTC [15144] LOG: shutting down
2026-06-20 09:23:13.621 UTC [15140] LOG: database system is shut down
2026-06-20T09:23:13Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45791 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:23:13 dist-test-slave-414w krb5kdc[16123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:23:13 dist-test-slave-414w krb5kdc[16123](info): set up 2 sockets
Jun 20 09:23:13 dist-test-slave-414w krb5kdc[16123](info): commencing operation
krb5kdc: starting...
W20260620 09:23:14.777436 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:23:14 dist-test-slave-414w krb5kdc[16123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:23:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:23:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.59.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.59.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.59.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/rangeradmin_127.4.59.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.59.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/rangeradmin_127.4.59.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.59.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.59.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.59.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/rangerlookup_127.4.59.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.59.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/rangerlookup_127.4.59.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.59.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.59.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.59.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/HTTP_127.4.59.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.59.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/HTTP_127.4.59.20@KRBTEST.COM.keytab.
I20260620 09:23:14.929311 4332 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task0Fx1M2/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/postgres -l logfile start
2026-06-20 09:23:17.379 UTC [16153] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 09:23:17.379 UTC [16153] LOG: listening on IPv4 address "127.4.59.20", port 54611
2026-06-20 09:23:17.382 UTC [16153] LOG: listening on Unix socket "/tmp/.s.PGSQL.54611"
2026-06-20 09:23:17.402 UTC [16156] LOG: database system was shut down at 2026-06-20 09:23:15 UTC
2026-06-20 09:23:17.406 UTC [16153] LOG: database system is ready to accept connections
I20260620 09:23:18.361290 4332 mini_postgres.cc:96] Postgres bound to 54611
2026-06-20 09:23:18.367 UTC [16164] FATAL: database "slave" does not exist
127.4.59.20:54611 - accepting connections
I20260620 09:23:18.367856 4332 mini_ranger.cc:161] Starting Ranger...
I20260620 09:23:18.382911 4332 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 09:23:18.844084 4332 mini_ranger.cc:87] Created ranger Postgres database
I20260620 09:23:18.844163 4332 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-admin
2026-06-20 09:23:19,109 [I] DB FLAVOR :POSTGRES
2026-06-20 09:23:19,110 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 09:23:19,110 [I] Checking connection..
2026-06-20 09:23:19,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 09:23:19,346 [I] Checking connection passed.
2026-06-20 09:23:19,347 [I] --------- Verifying version history table ---------
2026-06-20 09:23:19,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 09:23:19,611 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 09:23:19,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 09:23:19,854 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 09:23:19,854 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 09:23:19,854 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 09:23:20.105 UTC [16228] WARNING: there is no transaction in progress
2026-06-20 09:23:20,111 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 09:23:20,111 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 09:23:20,353 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 09:23:20,353 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 09:23:20,354 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 09:23:20,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 09:23:20,812 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-414w.c.gcp-upstream.internal','N') ;"
2026-06-20 09:23:21,034 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 09:23:21,261 [I] Table x_portal_user does not exist in database ranger
2026-06-20 09:23:21,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 09:23:21,496 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 09:23:21,496 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 09:23:21,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 09:23:21,978 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 09:23:21,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 09:23:22.934 UTC [16336] WARNING: there is no transaction in progress
2026-06-20 09:23:22.950 UTC [16336] WARNING: there is no transaction in progress
2026-06-20 09:23:22.966 UTC [16336] WARNING: there is no transaction in progress
2026-06-20 09:23:22.982 UTC [16336] WARNING: there is no transaction in progress
2026-06-20 09:23:22.998 UTC [16336] WARNING: there is no transaction in progress
2026-06-20 09:23:23.016 UTC [16336] WARNING: there is no transaction in progress
2026-06-20 09:23:23.071 UTC [16336] WARNING: there is no transaction in progress
2026-06-20 09:23:23.077 UTC [16336] WARNING: there is no transaction in progress
2026-06-20 09:23:23.091 UTC [16336] WARNING: there is no transaction in progress
2026-06-20 09:23:23.101 UTC [16336] WARNING: there is no transaction in progress
2026-06-20 09:23:23.110 UTC [16336] WARNING: there is no transaction in progress
2026-06-20 09:23:23.124 UTC [16336] WARNING: there is no transaction in progress
2026-06-20 09:23:23.133 UTC [16336] WARNING: there is no transaction in progress
2026-06-20 09:23:23.140 UTC [16336] WARNING: there is no transaction in progress
2026-06-20 09:23:23.146 UTC [16336] WARNING: there is no transaction in progress
2026-06-20 09:23:23.152 UTC [16336] WARNING: there is no transaction in progress
2026-06-20 09:23:23,797 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 09:23:23,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 09:23:24,043 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 09:23:24,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 09:23:24,280 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 09:23:24,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 09:23:24,524 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 09:23:24,524 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 09:23:24,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 09:23:24,970 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-414w.c.gcp-upstream.internal';"
2026-06-20 09:23:25,197 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 09:23:25,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 09:23:25,446 [I] DB_PATCHES have already been applied
I20260620 09:23:25.452397 4332 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-admin:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task0Fx1M2/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 09:23:25.452477 4332 mini_ranger.cc:193] Using host: 127.4.59.20
I20260620 09:23:25.455456 4332 mini_ranger.cc:227] Ranger admin URL: http://127.4.59.20:58381
Jun 20, 2026 9:23:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 9:23:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 9:23:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 9:23:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 9:23:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 9:23:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 9:23:26 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 09:23:26.577404 4332 mini_ranger.cc:160] Time spent starting Ranger: real 8.210s user 0.000s sys 0.008s
Jun 20, 2026 9:23:46 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 9:23:46 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 20, 2026 9:23:46 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 9:23:46 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 9:23:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 9:23:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 9:23:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 9:23:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 9:23:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 9:23:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 9:23:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 9:23:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 9:23:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 9:23:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 9:23:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 9:23:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 9:23:46 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 9:23:47 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 09:23:47.700439 4332 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.59.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.59.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.59.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/rangerkms_127.4.59.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.59.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/rangerkms_127.4.59.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.59.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.59.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.59.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/HTTP_127.4.59.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.59.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/HTTP_127.4.59.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 09:23:47 dist-test-slave-414w krb5kdc[16123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947427, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 09:23:47.783962 4332 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 09:23:47.793417 4332 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 09:23:48.689850 4332 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 09:23:48.689932 4332 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-kms
I20260620 09:23:48.689955 4332 mini_ranger_kms.cc:227] Using postgres at 127.4.59.20:54611
2026-06-20 09:23:48,854 [I] DB FLAVOR :POSTGRES
2026-06-20 09:23:48,854 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 09:23:48,854 [I] Checking connection
2026-06-20 09:23:48,854 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 09:23:49,084 [I] connection success
2026-06-20 09:23:49,084 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 09:23:49,085 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 09:23:49,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 09:23:49,314 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 09:23:49,315 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 09:23:49,315 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 09:23:49,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.59.20:54611/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 09:23:49,572 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 09:23:50.033843 4332 mini_ranger_kms.cc:310] Created kms service
I20260620 09:23:50.245204 4332 mini_ranger_kms.cc:326] Created kudu user
I20260620 09:23:50.285274 4332 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 09:23:50 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947427, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
I20260620 09:23:50.684336 4332 mini_ranger_kms.cc:384] Added ranger policy
I20260620 09:23:50.684523 4332 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-kms:/tmp/dist-test-task0Fx1M2/build/release/bin/postgresql.jar:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task0Fx1M2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 09:23:50.684541 4332 mini_ranger_kms.cc:240] Using host: 127.4.59.20
I20260620 09:23:50.688091 4332 mini_ranger_kms.cc:276] Ranger KMS PID: 16516
I20260620 09:23:50.688174 4332 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.59.20:41923
Jun 20, 2026 9:23:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 9:23:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 9:23:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 9:23:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 9:23:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 9:23:52 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260620 09:23:52.981281 4332 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.197s user 0.002s sys 0.145s
I20260620 09:23:52.981371 4332 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 09:23:52.981415 4332 mini_ranger_kms.cc:401] 127.4.59.20:41923/kms/v1/keys
Jun 20 09:23:53 dist-test-slave-414w krb5kdc[16123](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781947433, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.59.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 20 09:23:53 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947433, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.59.20@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
Jun 20 09:23:54 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947433, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.59.20@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
Jun 20 09:23:56 dist-test-slave-414w krb5kdc[16123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:23:56.337898 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:33517
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:33517
--ranger_config_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.59.20:41923/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:23:56.418337 16567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:23:56.418684 16567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:23:56.418716 16567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:23:56.419848 16567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:23:56.419888 16567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:23:56.419902 16567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:23:56.419915 16567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:23:56.419968 16567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:23:56.422430 16567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.59.20:41923/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:33517
--ranger_config_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:33517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:23:56.422703 16567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:23:56.422961 16567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:23:56.430130 16573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:56.430222 16575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:56.430121 16572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:23:56.430461 16567 server_base.cc:1034] running on GCE node
I20260620 09:23:56.430787 16567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:23:56.431139 16567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:23:56.432274 16567 hybrid_clock.cc:648] HybridClock initialized: now 1781947436432258 us; error 34 us; skew 500 ppm
Jun 20 09:23:56 dist-test-slave-414w krb5kdc[16123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947436, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:23:56.435448 16567 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:23:56.435943 16567 webserver.cc:460] Webserver started at http://127.4.59.62:40875/ using document root <none> and password file <none>
I20260620 09:23:56.436224 16567 fs_manager.cc:359] Metadata directory not provided
I20260620 09:23:56.436297 16567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:23:56.436460 16567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 09:23:56 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947436, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
I20260620 09:23:56.502501 16567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "f73066edab2b42f0b8d3ca4fd3aefa34"
format_stamp: "Formatted at 2026-06-20 09:23:56 on dist-test-slave-414w"
server_key: "de0752b8d54669c67c4f0a2b7ba45071"
server_key_iv: "11e99c234d982802b3fe699e3fd6ca5c"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:56.502903 16567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "f73066edab2b42f0b8d3ca4fd3aefa34"
format_stamp: "Formatted at 2026-06-20 09:23:56 on dist-test-slave-414w"
server_key: "de0752b8d54669c67c4f0a2b7ba45071"
server_key_iv: "11e99c234d982802b3fe699e3fd6ca5c"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:56.504304 16567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:23:56.519418 16582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:56.519611 16567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:23:56.519704 16567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "f73066edab2b42f0b8d3ca4fd3aefa34"
format_stamp: "Formatted at 2026-06-20 09:23:56 on dist-test-slave-414w"
server_key: "de0752b8d54669c67c4f0a2b7ba45071"
server_key_iv: "11e99c234d982802b3fe699e3fd6ca5c"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:56.519825 16567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:23:56.533730 16567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:23:56.535809 16567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:23:56.535959 16567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:23:56.540951 16567 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:33517
I20260620 09:23:56.540977 16634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:33517 every 8 connection(s)
I20260620 09:23:56.541390 16567 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:23:56.542146 16635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:23:56.543045 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 16567
I20260620 09:23:56.543152 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:23:56.543344 4332 external_mini_cluster.cc:1428] Setting key f42d7892ff6c43ec56652001518e7a5b
I20260620 09:23:56.545500 16635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34: Bootstrap starting.
I20260620 09:23:56.546228 16635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34: Neither blocks nor log segments found. Creating new log.
I20260620 09:23:56.546535 16635 log.cc:824] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34: Log is configured to *not* fsync() on all Append() calls
I20260620 09:23:56.547430 16635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34: No bootstrap required, opened a new log
Jun 20 09:23:56 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:23:56.550031 16635 raft_consensus.cc:348] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73066edab2b42f0b8d3ca4fd3aefa34" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33517 } }
I20260620 09:23:56.550209 16635 raft_consensus.cc:374] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:23:56.550276 16635 raft_consensus.cc:729] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f73066edab2b42f0b8d3ca4fd3aefa34, State: Initialized, Role: FOLLOWER
I20260620 09:23:56.550436 16635 consensus_queue.cc:260] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73066edab2b42f0b8d3ca4fd3aefa34" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33517 } }
I20260620 09:23:56.550536 16635 raft_consensus.cc:388] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:23:56.550585 16635 raft_consensus.cc:482] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:23:56.550633 16635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:23:56.551324 16635 raft_consensus.cc:504] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73066edab2b42f0b8d3ca4fd3aefa34" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33517 } }
I20260620 09:23:56.551432 16635 leader_election.cc:302] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f73066edab2b42f0b8d3ca4fd3aefa34; no voters:
I20260620 09:23:56.551631 16635 leader_election.cc:288] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:23:56.551671 16640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:23:56.551899 16635 sys_catalog.cc:564] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:23:56.552706 16640 raft_consensus.cc:686] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 1 LEADER]: Becoming Leader. State: Replica: f73066edab2b42f0b8d3ca4fd3aefa34, State: Running, Role: LEADER
I20260620 09:23:56.552870 16640 consensus_queue.cc:237] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73066edab2b42f0b8d3ca4fd3aefa34" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33517 } }
I20260620 09:23:56.554272 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:56.543894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49184 (local address 127.4.59.62:33517)
0620 09:23:56.544105 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:56.544113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:56.544136 (+ 23us) server_negotiation.cc:407] Connection header received
0620 09:23:56.544564 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:56.544569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:56.544614 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:56.544762 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:23:56.545457 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:56.546484 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:56.547489 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:56.547909 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:56.548039 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:56.550696 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:23:56.550700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:23:56.550701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:23:56.550713 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:23:56.552595 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:56.552971 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:56.552973 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:56.552974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:56.553071 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:56.553344 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:56.553345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:56.553345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:56.553599 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:23:56.553764 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:56.553945 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 09:23:56.554070 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":70,"threads_started":1}
I20260620 09:23:56.554304 16641 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f73066edab2b42f0b8d3ca4fd3aefa34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73066edab2b42f0b8d3ca4fd3aefa34" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33517 } } }
I20260620 09:23:56.554409 16641 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [sys.catalog]: This master's current role is: LEADER
I20260620 09:23:56.554659 16644 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f73066edab2b42f0b8d3ca4fd3aefa34. Latest consensus state: current_term: 1 leader_uuid: "f73066edab2b42f0b8d3ca4fd3aefa34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73066edab2b42f0b8d3ca4fd3aefa34" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33517 } } }
I20260620 09:23:56.554728 16644 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [sys.catalog]: This master's current role is: LEADER
I20260620 09:23:56.554898 16635 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 09:23:56.555320 16654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:23:56.558102 16654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:23:56.560973 16654 catalog_manager.cc:1347] Generated new cluster ID: 6698287783574970b521381f4059a28f
I20260620 09:23:56.561026 16654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:23:56.576193 16654 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:23:56.577067 16654 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:23:56.595067 16654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34: Generated new TSK 0
I20260620 09:23:56.595408 16654 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:23:57.287587 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.59.20:41923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:33517
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:23:57.402446 16680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:23:57.402678 16680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:23:57.402715 16680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:23:57.403891 16680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:23:57.403960 16680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:23:57.404006 16680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:23:57.406270 16680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.59.20:41923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:33517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:23:57.406802 16680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:23:57.407145 16680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:23:57.413244 16687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:57.413271 16690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:57.413410 16688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:23:57.413884 16680 server_base.cc:1034] running on GCE node
I20260620 09:23:57.414137 16680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:23:57.414398 16680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:23:57.415567 16680 hybrid_clock.cc:648] HybridClock initialized: now 1781947437415556 us; error 33 us; skew 500 ppm
Jun 20 09:23:57 dist-test-slave-414w krb5kdc[16123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947437, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:23:57.418419 16680 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:23:57.418949 16680 webserver.cc:460] Webserver started at http://127.4.59.1:35615/ using document root <none> and password file <none>
I20260620 09:23:57.419236 16680 fs_manager.cc:359] Metadata directory not provided
I20260620 09:23:57.419332 16680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:23:57.419523 16680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 09:23:57 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947437, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
I20260620 09:23:57.436573 16680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "c85d4a8d9dce4297844500a6e48de248"
format_stamp: "Formatted at 2026-06-20 09:23:57 on dist-test-slave-414w"
server_key: "fc8b2406ef12d88506b0cabc05f534f9"
server_key_iv: "7b7654514df7f01e4bcbb44451a4f31d"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:57.437027 16680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "c85d4a8d9dce4297844500a6e48de248"
format_stamp: "Formatted at 2026-06-20 09:23:57 on dist-test-slave-414w"
server_key: "fc8b2406ef12d88506b0cabc05f534f9"
server_key_iv: "7b7654514df7f01e4bcbb44451a4f31d"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:57.441546 16680 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260620 09:23:57.457070 16697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:57.457367 16680 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:23:57.457474 16680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "c85d4a8d9dce4297844500a6e48de248"
format_stamp: "Formatted at 2026-06-20 09:23:57 on dist-test-slave-414w"
server_key: "fc8b2406ef12d88506b0cabc05f534f9"
server_key_iv: "7b7654514df7f01e4bcbb44451a4f31d"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:57.457566 16680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:23:57.473119 16680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:23:57.475638 16680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:23:57.475834 16680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:23:57.476123 16680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:23:57.476531 16680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:23:57.476625 16680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:57.476723 16680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:23:57.476794 16680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:57.483348 16680 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:36565
I20260620 09:23:57.483817 16680 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:23:57.487293 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 16680
I20260620 09:23:57.487396 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:23:57.487574 4332 external_mini_cluster.cc:1428] Setting key d6a10e2cc538f2af2c9ae0962fdf1ed3
I20260620 09:23:57.485888 16810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:36565 every 8 connection(s)
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
Jun 20 09:23:57 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947437, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:23:57.510210 16813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:57.486472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:33517 (local address 127.4.59.1:48643)
0620 09:23:57.486713 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:57.486744 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:57.487399 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:57.487794 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:57.487799 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:57.488056 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:23:57.488335 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:57.488342 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:57.489367 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:57.489368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:57.490709 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:57.490712 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:57.491307 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:57.491308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:57.491455 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:57.492070 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:23:57.492081 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:23:57.506255 (+ 14174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:23:57.508468 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:57.508480 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:57.508483 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:57.508799 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:57.509014 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:57.509015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:57.509015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:57.509173 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:57.509481 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:23:57.509488 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:23:57.509682 (+ 194us) client_negotiation.cc:769] Sending connection context
0620 09:23:57.509797 (+ 115us) client_negotiation.cc:240] Negotiation successful
0620 09:23:57.509965 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":83,"threads_started":1}
I20260620 09:23:57.510766 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:57.486479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:48643 (local address 127.4.59.62:33517)
0620 09:23:57.486623 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:57.486625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:57.486830 (+ 205us) server_negotiation.cc:407] Connection header received
0620 09:23:57.487574 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:57.487575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:57.487592 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:57.487660 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:23:57.488441 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:57.489263 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:57.490811 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:57.491161 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:57.491223 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:57.506416 (+ 15193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:23:57.506419 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:23:57.506419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:23:57.506429 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:23:57.508364 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:57.508885 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:57.508886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:57.508887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:57.508949 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:57.509246 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:57.509247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:57.509247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:57.509395 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:23:57.509455 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:57.510566 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0620 09:23:57.510661 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260620 09:23:57.511341 16811 heartbeater.cc:347] Connected to a master server at 127.4.59.62:33517
I20260620 09:23:57.511438 16811 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:57.511620 16811 heartbeater.cc:511] Master 127.4.59.62:33517 requested a full tablet report, sending...
I20260620 09:23:57.512142 16599 ts_manager.cc:194] Registered new tserver with Master: c85d4a8d9dce4297844500a6e48de248 (127.4.59.1:36565)
I20260620 09:23:57.513034 16599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:48643
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:23:57.523960 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.59.20:41923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:33517
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 09:23:57 dist-test-slave-414w krb5kdc[16123](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781947437, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 09:23:57.615085 16817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:23:57.615294 16817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:23:57.615358 16817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:23:57.617206 16817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:23:57.617286 16817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:23:57.617339 16817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:23:57.619148 16817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.59.20:41923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:33517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:23:57.619529 16817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:23:57.619807 16817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:23:57.625895 16824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:57.626132 16825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:57.626339 16827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:23:57.627420 16817 server_base.cc:1034] running on GCE node
I20260620 09:23:57.627720 16817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:23:57.627982 16817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:23:57.629172 16817 hybrid_clock.cc:648] HybridClock initialized: now 1781947437629152 us; error 32 us; skew 500 ppm
Jun 20 09:23:57 dist-test-slave-414w krb5kdc[16123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947437, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:23:57.632112 16817 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:23:57.632584 16817 webserver.cc:460] Webserver started at http://127.4.59.2:41667/ using document root <none> and password file <none>
I20260620 09:23:57.632845 16817 fs_manager.cc:359] Metadata directory not provided
I20260620 09:23:57.632896 16817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:23:57.633013 16817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 09:23:57 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947437, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
I20260620 09:23:57.653888 16817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "3b74a34f762e410aac3831fa6b620f57"
format_stamp: "Formatted at 2026-06-20 09:23:57 on dist-test-slave-414w"
server_key: "58cd81275e2c909a7649c1d504e1a236"
server_key_iv: "8f88b6e4f280bb84c49292cffc6b1513"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:57.654338 16817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "3b74a34f762e410aac3831fa6b620f57"
format_stamp: "Formatted at 2026-06-20 09:23:57 on dist-test-slave-414w"
server_key: "58cd81275e2c909a7649c1d504e1a236"
server_key_iv: "8f88b6e4f280bb84c49292cffc6b1513"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:57.659446 16817 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260620 09:23:57.671577 16834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:57.671878 16817 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:23:57.671974 16817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "3b74a34f762e410aac3831fa6b620f57"
format_stamp: "Formatted at 2026-06-20 09:23:57 on dist-test-slave-414w"
server_key: "58cd81275e2c909a7649c1d504e1a236"
server_key_iv: "8f88b6e4f280bb84c49292cffc6b1513"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:57.672065 16817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:23:57.691949 16817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:23:57.694487 16817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:23:57.694700 16817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:23:57.694993 16817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:23:57.695365 16817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:23:57.695449 16817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:57.695518 16817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:23:57.695573 16817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:57.703097 16817 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:34789
I20260620 09:23:57.703586 16817 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:23:57.706065 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 16817
I20260620 09:23:57.706156 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:23:57.706365 4332 external_mini_cluster.cc:1428] Setting key 72e7ab0d7406bab05c63ebff2ecb881c
I20260620 09:23:57.706262 16948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:34789 every 8 connection(s)
Jun 20 09:23:57 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947437, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:23:57.718096 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:57.705838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:33851 (local address 127.4.59.62:33517)
0620 09:23:57.705887 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:57.705889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:57.707215 (+ 1326us) server_negotiation.cc:407] Connection header received
0620 09:23:57.707914 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:57.707915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:57.707927 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:57.707958 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:23:57.708798 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:57.709696 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:57.711323 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:57.711617 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:57.711666 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:57.714596 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:23:57.714598 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:23:57.714598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:23:57.714606 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:23:57.716375 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:57.716891 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:57.716892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:57.716893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:57.716955 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:57.717311 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:57.717312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:57.717313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:57.717451 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:23:57.717496 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:57.717903 (+ 407us) server_negotiation.cc:299] Negotiation successful
0620 09:23:57.717997 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 09:23:57.718868 16951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:57.705550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:33517 (local address 127.4.59.2:33851)
0620 09:23:57.706583 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:57.706625 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:57.707777 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:57.708140 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:57.708144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:57.708395 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:23:57.708670 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:57.708676 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:57.709916 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:57.709917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:57.711208 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:57.711211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:57.711779 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:57.711780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:57.711917 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:57.712522 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:23:57.712532 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:23:57.714478 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:23:57.716473 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:57.716480 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:57.716482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:57.716792 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:57.717032 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:57.717033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:57.717033 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:57.717219 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:57.717555 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:23:57.717561 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:23:57.717738 (+ 177us) client_negotiation.cc:769] Sending connection context
0620 09:23:57.718453 (+ 715us) client_negotiation.cc:240] Negotiation successful
0620 09:23:57.718636 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"thread_start_us":79,"threads_started":1}
I20260620 09:23:57.719544 16949 heartbeater.cc:347] Connected to a master server at 127.4.59.62:33517
I20260620 09:23:57.719637 16949 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:57.719825 16949 heartbeater.cc:511] Master 127.4.59.62:33517 requested a full tablet report, sending...
I20260620 09:23:57.720256 16599 ts_manager.cc:194] Registered new tserver with Master: 3b74a34f762e410aac3831fa6b620f57 (127.4.59.2:34789)
I20260620 09:23:57.720732 16599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:33851
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:23:57.752291 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.59.20:41923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:33517
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:23:57.886960 16954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:23:57.887320 16954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:23:57.887418 16954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:23:57.889192 16954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:23:57.889401 16954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:23:57.889508 16954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:23:57.891739 16954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.59.20:41923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:33517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:23:57.892227 16954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:23:57.892542 16954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:23:57.899811 16960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:23:57.899998 16954 server_base.cc:1034] running on GCE node
W20260620 09:23:57.900233 16959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:57.900435 16962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:23:57.900698 16954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:23:57.900988 16954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:23:57.902158 16954 hybrid_clock.cc:648] HybridClock initialized: now 1781947437902131 us; error 42 us; skew 500 ppm
Jun 20 09:23:57 dist-test-slave-414w krb5kdc[16123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947437, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:23:57.905321 16954 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:23:57.905822 16954 webserver.cc:460] Webserver started at http://127.4.59.3:40015/ using document root <none> and password file <none>
I20260620 09:23:57.906086 16954 fs_manager.cc:359] Metadata directory not provided
I20260620 09:23:57.906153 16954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:23:57.906282 16954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 09:23:57 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947437, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
I20260620 09:23:57.928264 16954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "599482cac8f1486e8e8221cb147563a6"
format_stamp: "Formatted at 2026-06-20 09:23:57 on dist-test-slave-414w"
server_key: "2d5f4024b3320d26f4e0e1851696d5f4"
server_key_iv: "bb620b023f1eefd61d3b5a02a8ce741a"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:57.929811 16954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "599482cac8f1486e8e8221cb147563a6"
format_stamp: "Formatted at 2026-06-20 09:23:57 on dist-test-slave-414w"
server_key: "2d5f4024b3320d26f4e0e1851696d5f4"
server_key_iv: "bb620b023f1eefd61d3b5a02a8ce741a"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:57.931813 16954 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260620 09:23:57.950846 16969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:57.951071 16954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 09:23:57.951156 16954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "599482cac8f1486e8e8221cb147563a6"
format_stamp: "Formatted at 2026-06-20 09:23:57 on dist-test-slave-414w"
server_key: "2d5f4024b3320d26f4e0e1851696d5f4"
server_key_iv: "bb620b023f1eefd61d3b5a02a8ce741a"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:57.951244 16954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:23:57.978677 16954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:23:57.987234 16954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:23:57.987479 16954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:23:57.987766 16954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:23:57.988186 16954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:23:57.988277 16954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:57.988354 16954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:23:57.988413 16954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:57.998737 16954 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:38527
I20260620 09:23:57.999238 16954 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:23:58.000094 17082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:38527 every 8 connection(s)
Jun 20 09:23:58 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947437, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:23:58.008270 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 16954
I20260620 09:23:58.008371 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:23:58.008565 4332 external_mini_cluster.cc:1428] Setting key 07756a0e9918270cdecacbaf3cbcffde
I20260620 09:23:58.011971 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:58.001012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:54039 (local address 127.4.59.62:33517)
0620 09:23:58.001054 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:58.001056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:58.001834 (+ 778us) server_negotiation.cc:407] Connection header received
0620 09:23:58.002560 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:58.002562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:58.002574 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:58.002604 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:23:58.003388 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.004166 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.005641 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.005934 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.005974 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:58.008540 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:23:58.008542 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:23:58.008542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:23:58.008549 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:23:58.010504 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:58.010968 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:58.010969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:58.010969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:58.011023 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:58.011274 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:58.011275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:58.011275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:58.011396 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:23:58.011434 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:58.011791 (+ 357us) server_negotiation.cc:299] Negotiation successful
0620 09:23:58.011875 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:23:58.012666 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:58.000750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:33517 (local address 127.4.59.3:54039)
0620 09:23:58.001684 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:58.001733 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:58.002424 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:58.002782 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:58.002785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:58.003023 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:23:58.003284 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:58.003290 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.004280 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.004280 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:58.005545 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:58.005547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.006073 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.006074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:58.006216 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:58.006763 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:23:58.006773 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:23:58.008385 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:23:58.010612 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:58.010618 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:58.010620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:58.010887 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:58.011090 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:58.011091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:58.011091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:58.011208 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:58.011474 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:23:58.011479 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:23:58.011648 (+ 169us) client_negotiation.cc:769] Sending connection context
0620 09:23:58.012282 (+ 634us) client_negotiation.cc:240] Negotiation successful
0620 09:23:58.012432 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":890,"thread_start_us":79,"threads_started":1}
I20260620 09:23:58.013355 17083 heartbeater.cc:347] Connected to a master server at 127.4.59.62:33517
I20260620 09:23:58.013437 17083 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:58.013612 17083 heartbeater.cc:511] Master 127.4.59.62:33517 requested a full tablet report, sending...
I20260620 09:23:58.013970 16599 ts_manager.cc:194] Registered new tserver with Master: 599482cac8f1486e8e8221cb147563a6 (127.4.59.3:38527)
I20260620 09:23:58.014434 16599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:54039
I20260620 09:23:58.020560 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:23:58.021281 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 16680
I20260620 09:23:58.028851 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 16817
I20260620 09:23:58.035620 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 16954
I20260620 09:23:58.041471 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 16567
I20260620 09:23:58.047339 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:33517
--webserver_interface=127.4.59.62
--webserver_port=40875
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:33517
--ranger_config_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.59.20:41923/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.59.20:41923/kms with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:23:58.130430 17086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:23:58.130668 17086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:23:58.130707 17086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:23:58.131939 17086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:23:58.131994 17086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:23:58.132017 17086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:23:58.132048 17086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:23:58.132076 17086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:23:58.133693 17086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.59.20:41923/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:33517
--ranger_config_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:33517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=40875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:23:58.133950 17086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:23:58.134163 17086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:23:58.139717 17092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:58.139724 17094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:58.139787 17091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:23:58.140081 17086 server_base.cc:1034] running on GCE node
I20260620 09:23:58.140328 17086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:23:58.140623 17086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:23:58.141780 17086 hybrid_clock.cc:648] HybridClock initialized: now 1781947438141767 us; error 27 us; skew 500 ppm
Jun 20 09:23:58 dist-test-slave-414w krb5kdc[16123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947438, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:23:58.144644 17086 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:23:58.145160 17086 webserver.cc:460] Webserver started at http://127.4.59.62:40875/ using document root <none> and password file <none>
I20260620 09:23:58.145392 17086 fs_manager.cc:359] Metadata directory not provided
I20260620 09:23:58.145457 17086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 09:23:58 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947438, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
I20260620 09:23:58.191905 17086 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:23:58.192782 17102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:58.193022 17086 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:23:58.193127 17086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "f73066edab2b42f0b8d3ca4fd3aefa34"
format_stamp: "Formatted at 2026-06-20 09:23:56 on dist-test-slave-414w"
server_key: "de0752b8d54669c67c4f0a2b7ba45071"
server_key_iv: "11e99c234d982802b3fe699e3fd6ca5c"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:58.193604 17086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:23:58.237772 17086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:23:58.239610 17086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:23:58.239760 17086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:23:58.244165 17086 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:33517
I20260620 09:23:58.244204 17154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:33517 every 8 connection(s)
I20260620 09:23:58.244556 17086 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:23:58.245815 17155 sys_catalog.cc:263] Verifying existing consensus state
I20260620 09:23:58.246376 17155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34: Bootstrap starting.
I20260620 09:23:58.253629 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 17086
I20260620 09:23:58.253895 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.59.20:41923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:36565
--local_ip_for_outbound_sockets=127.4.59.1
--tserver_master_addrs=127.4.59.62:33517
--webserver_port=35615
--webserver_interface=127.4.59.1
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.59.20:41923/kms with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:23:58.254133 17155 log.cc:824] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34: Log is configured to *not* fsync() on all Append() calls
I20260620 09:23:58.256147 17155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 09:23:58.256440 17155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34: Bootstrap complete.
I20260620 09:23:58.258874 17155 raft_consensus.cc:348] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73066edab2b42f0b8d3ca4fd3aefa34" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33517 } }
I20260620 09:23:58.259099 17155 raft_consensus.cc:729] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f73066edab2b42f0b8d3ca4fd3aefa34, State: Initialized, Role: FOLLOWER
I20260620 09:23:58.259250 17155 consensus_queue.cc:260] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73066edab2b42f0b8d3ca4fd3aefa34" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33517 } }
I20260620 09:23:58.259330 17155 raft_consensus.cc:388] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:23:58.259380 17155 raft_consensus.cc:482] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:23:58.259426 17155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 1 FOLLOWER]: Advancing to term 2
I20260620 09:23:58.260416 17155 raft_consensus.cc:504] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73066edab2b42f0b8d3ca4fd3aefa34" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33517 } }
I20260620 09:23:58.260504 17155 leader_election.cc:302] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f73066edab2b42f0b8d3ca4fd3aefa34; no voters:
I20260620 09:23:58.260684 17155 leader_election.cc:288] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 09:23:58.260716 17159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 09:23:58.260833 17159 raft_consensus.cc:686] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [term 2 LEADER]: Becoming Leader. State: Replica: f73066edab2b42f0b8d3ca4fd3aefa34, State: Running, Role: LEADER
I20260620 09:23:58.260928 17159 consensus_queue.cc:237] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73066edab2b42f0b8d3ca4fd3aefa34" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33517 } }
I20260620 09:23:58.260934 17155 sys_catalog.cc:564] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:23:58.261190 17161 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f73066edab2b42f0b8d3ca4fd3aefa34. Latest consensus state: current_term: 2 leader_uuid: "f73066edab2b42f0b8d3ca4fd3aefa34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73066edab2b42f0b8d3ca4fd3aefa34" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33517 } } }
I20260620 09:23:58.261257 17161 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [sys.catalog]: This master's current role is: LEADER
I20260620 09:23:58.262033 17163 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:23:58.262574 17163 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:23:58.262603 17160 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f73066edab2b42f0b8d3ca4fd3aefa34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73066edab2b42f0b8d3ca4fd3aefa34" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33517 } } }
I20260620 09:23:58.262709 17163 catalog_manager.cc:1259] Loaded cluster ID: 6698287783574970b521381f4059a28f
I20260620 09:23:58.263083 17160 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34 [sys.catalog]: This master's current role is: LEADER
I20260620 09:23:58.263096 17163 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:23:58.262938 17155 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 09:23:58.264024 17163 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:23:58.264204 17163 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34: Loaded TSK: 0
I20260620 09:23:58.264405 17163 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 09:23:58.366304 17158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:23:58.366583 17158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:23:58.366632 17158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:23:58.368263 17158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:23:58.368324 17158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:23:58.368371 17158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:23:58.370499 17158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.59.20:41923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:36565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=35615
--tserver_master_addrs=127.4.59.62:33517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:23:58.370754 17158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:23:58.371019 17158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:23:58.378461 17193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:58.378484 17192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:58.378680 17195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:23:58.378762 17158 server_base.cc:1034] running on GCE node
I20260620 09:23:58.378965 17158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:23:58.379244 17158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:23:58.380443 17158 hybrid_clock.cc:648] HybridClock initialized: now 1781947438380426 us; error 49 us; skew 500 ppm
Jun 20 09:23:58 dist-test-slave-414w krb5kdc[16123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947438, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:23:58.383576 17158 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:23:58.384099 17158 webserver.cc:460] Webserver started at http://127.4.59.1:35615/ using document root <none> and password file <none>
I20260620 09:23:58.384351 17158 fs_manager.cc:359] Metadata directory not provided
I20260620 09:23:58.384433 17158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 09:23:58 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947438, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
I20260620 09:23:58.402658 17158 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260620 09:23:58.403779 17203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:58.403981 17158 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 09:23:58.404063 17158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "c85d4a8d9dce4297844500a6e48de248"
format_stamp: "Formatted at 2026-06-20 09:23:57 on dist-test-slave-414w"
server_key: "fc8b2406ef12d88506b0cabc05f534f9"
server_key_iv: "7b7654514df7f01e4bcbb44451a4f31d"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:58.404523 17158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:23:58.422484 17158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:23:58.424780 17158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:23:58.424996 17158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:23:58.425274 17158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:23:58.425889 17158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:23:58.425984 17158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:58.426093 17158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:23:58.426172 17158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:58.433384 17158 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:36565
I20260620 09:23:58.433907 17158 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:23:58.433887 17316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:36565 every 8 connection(s)
I20260620 09:23:58.439662 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 17158
I20260620 09:23:58.439956 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.59.20:41923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:34789
--local_ip_for_outbound_sockets=127.4.59.2
--tserver_master_addrs=127.4.59.62:33517
--webserver_port=41667
--webserver_interface=127.4.59.2
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.59.20:41923/kms with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 09:23:58 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947438, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:23:58.449572 17320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:58.434786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:49079 (local address 127.4.59.62:33517)
0620 09:23:58.435009 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:58.435019 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:58.435284 (+ 265us) server_negotiation.cc:407] Connection header received
0620 09:23:58.436044 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:58.436052 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:58.436110 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:58.436258 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:23:58.436955 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.438426 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.441965 (+ 3539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.442596 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.442697 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:58.445410 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:23:58.445414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:23:58.445414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:23:58.445427 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:23:58.447504 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:58.448017 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:58.448020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:58.448021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:58.448128 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:58.448400 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:58.448402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:58.448402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:58.448696 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:23:58.448773 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:58.449230 (+ 457us) server_negotiation.cc:299] Negotiation successful
0620 09:23:58.449348 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":92,"threads_started":1}
I20260620 09:23:58.450103 17319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:58.434699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:33517 (local address 127.4.59.1:49079)
0620 09:23:58.435109 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:58.435152 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:58.435767 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:58.436328 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:58.436333 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:58.436553 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:23:58.436811 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:58.436817 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.438578 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.438579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:58.441861 (+ 3282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:58.441864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.442864 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.442865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:58.442990 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:58.443649 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:23:58.443659 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:23:58.445275 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:23:58.447602 (+ 2327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:58.447608 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:58.447611 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:58.447917 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:58.448200 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:58.448201 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:58.448201 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:58.448326 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:58.448815 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:23:58.448821 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:23:58.448968 (+ 147us) client_negotiation.cc:769] Sending connection context
0620 09:23:58.449068 (+ 100us) client_negotiation.cc:240] Negotiation successful
0620 09:23:58.449882 (+ 814us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":91,"threads_started":1}
I20260620 09:23:58.450845 17317 heartbeater.cc:347] Connected to a master server at 127.4.59.62:33517
I20260620 09:23:58.450941 17317 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:58.451120 17317 heartbeater.cc:511] Master 127.4.59.62:33517 requested a full tablet report, sending...
W20260620 09:23:58.452386 17317 heartbeater.cc:644] Failed to heartbeat to 127.4.59.62:33517 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 09:23:58.452454 17317 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:58.452514 17317 heartbeater.cc:511] Master 127.4.59.62:33517 requested a full tablet report, sending...
I20260620 09:23:58.452908 17317 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:58.452981 17317 heartbeater.cc:511] Master 127.4.59.62:33517 requested a full tablet report, sending...
W20260620 09:23:58.453342 17317 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 09:23:58.565220 17321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:23:58.565465 17321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:23:58.565493 17321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:23:58.567085 17321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:23:58.567144 17321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:23:58.567183 17321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:23:58.569466 17321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.59.20:41923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:34789
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=41667
--tserver_master_addrs=127.4.59.62:33517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:23:58.569702 17321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:23:58.569978 17321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:23:58.575452 17329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:58.575551 17326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:58.575445 17327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:23:58.575872 17321 server_base.cc:1034] running on GCE node
I20260620 09:23:58.576066 17321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:23:58.576329 17321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:23:58.577491 17321 hybrid_clock.cc:648] HybridClock initialized: now 1781947438577477 us; error 25 us; skew 500 ppm
Jun 20 09:23:58 dist-test-slave-414w krb5kdc[16123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947438, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:23:58.579941 17321 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:23:58.580364 17321 webserver.cc:460] Webserver started at http://127.4.59.2:41667/ using document root <none> and password file <none>
I20260620 09:23:58.580595 17321 fs_manager.cc:359] Metadata directory not provided
I20260620 09:23:58.580675 17321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 09:23:58 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947438, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
I20260620 09:23:58.605242 17321 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260620 09:23:58.606446 17337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:58.606626 17321 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:23:58.606721 17321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "3b74a34f762e410aac3831fa6b620f57"
format_stamp: "Formatted at 2026-06-20 09:23:57 on dist-test-slave-414w"
server_key: "58cd81275e2c909a7649c1d504e1a236"
server_key_iv: "8f88b6e4f280bb84c49292cffc6b1513"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:58.607232 17321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:23:58.635596 17321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:23:58.638098 17321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:23:58.638293 17321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:23:58.638549 17321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:23:58.638942 17321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:23:58.638996 17321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:58.639030 17321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:23:58.639050 17321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:58.646499 17321 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:34789
I20260620 09:23:58.646585 17450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:34789 every 8 connection(s)
I20260620 09:23:58.647042 17321 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
Jun 20 09:23:58 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947438, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:23:58.656350 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 17321
I20260620 09:23:58.656638 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.59.20:41923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:38527
--local_ip_for_outbound_sockets=127.4.59.3
--tserver_master_addrs=127.4.59.62:33517
--webserver_port=40015
--webserver_interface=127.4.59.3
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.59.20:41923/kms with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:23:58.657101 17453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:58.647836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:33517 (local address 127.4.59.2:38447)
0620 09:23:58.648035 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:58.648081 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:58.648718 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:58.649344 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:58.649348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:58.649567 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:23:58.649835 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:58.649839 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.650530 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.650531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:58.651386 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:58.651388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.651808 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.651810 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:58.651929 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:58.652484 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:23:58.652492 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:23:58.653932 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:23:58.655537 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:58.655544 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:58.655546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:58.655762 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:58.655983 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:58.655984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:58.655984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:58.656092 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:58.656402 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:23:58.656408 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:23:58.656568 (+ 160us) client_negotiation.cc:769] Sending connection context
0620 09:23:58.656704 (+ 136us) client_negotiation.cc:240] Negotiation successful
0620 09:23:58.656867 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":97,"threads_started":1}
I20260620 09:23:58.657219 17320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:58.649029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:38447 (local address 127.4.59.62:33517)
0620 09:23:58.649074 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:58.649076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:58.649087 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:23:58.649118 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:58.649118 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:58.649133 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:58.649219 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:23:58.649927 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.650438 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.651479 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.651706 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.651784 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:58.654022 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:23:58.654024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:23:58.654024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:23:58.654031 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:23:58.655436 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:58.655853 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:58.655854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:58.655855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:58.655916 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:58.656160 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:58.656161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:58.656161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:58.656306 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:23:58.656379 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:58.656969 (+ 590us) server_negotiation.cc:299] Negotiation successful
0620 09:23:58.657062 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 09:23:58.658099 17451 heartbeater.cc:347] Connected to a master server at 127.4.59.62:33517
I20260620 09:23:58.658231 17451 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:58.658404 17451 heartbeater.cc:511] Master 127.4.59.62:33517 requested a full tablet report, sending...
W20260620 09:23:58.658998 17451 heartbeater.cc:644] Failed to heartbeat to 127.4.59.62:33517 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 09:23:58.659078 17451 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:58.659148 17451 heartbeater.cc:511] Master 127.4.59.62:33517 requested a full tablet report, sending...
I20260620 09:23:58.659538 17451 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:58.659613 17451 heartbeater.cc:511] Master 127.4.59.62:33517 requested a full tablet report, sending...
W20260620 09:23:58.659941 17451 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 09:23:58.752079 17454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:23:58.752319 17454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:23:58.752355 17454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:23:58.753686 17454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:23:58.753777 17454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:23:58.753835 17454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:23:58.756162 17454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.59.20:41923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:38527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=40015
--tserver_master_addrs=127.4.59.62:33517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:23:58.756620 17454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:23:58.756942 17454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:23:58.764657 17461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:58.765319 17460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:23:58.767807 17463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:23:58.768607 17454 server_base.cc:1034] running on GCE node
I20260620 09:23:58.768863 17454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:23:58.769160 17454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:23:58.770349 17454 hybrid_clock.cc:648] HybridClock initialized: now 1781947438770334 us; error 34 us; skew 500 ppm
Jun 20 09:23:58 dist-test-slave-414w krb5kdc[16123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947438, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:23:58.773483 17454 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:23:58.773990 17454 webserver.cc:460] Webserver started at http://127.4.59.3:40015/ using document root <none> and password file <none>
I20260620 09:23:58.774251 17454 fs_manager.cc:359] Metadata directory not provided
I20260620 09:23:58.774307 17454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 09:23:58 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947438, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
I20260620 09:23:58.788287 17454 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:23:58.789662 17471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:58.789919 17454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:23:58.790021 17454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "599482cac8f1486e8e8221cb147563a6"
format_stamp: "Formatted at 2026-06-20 09:23:57 on dist-test-slave-414w"
server_key: "2d5f4024b3320d26f4e0e1851696d5f4"
server_key_iv: "bb620b023f1eefd61d3b5a02a8ce741a"
server_key_version: "kuduclusterkey@0"
I20260620 09:23:58.790513 17454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:23:58.805214 17454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:23:58.807420 17454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:23:58.807577 17454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:23:58.807811 17454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:23:58.808145 17454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:23:58.808187 17454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:58.808228 17454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:23:58.808254 17454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:23:58.815270 17454 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:38527
I20260620 09:23:58.815641 17454 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:23:58.816022 17584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:38527 every 8 connection(s)
Jun 20 09:23:58 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947438, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:23:58.823813 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 17454
I20260620 09:23:58.828717 17320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:58.816368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:43077 (local address 127.4.59.62:33517)
0620 09:23:58.816418 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:58.816421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:58.816777 (+ 356us) server_negotiation.cc:407] Connection header received
0620 09:23:58.817334 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:58.817335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:58.817348 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:58.817421 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:23:58.818190 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.818910 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.820077 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.820374 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.820451 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:58.825586 (+ 5135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:23:58.825587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:23:58.825588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:23:58.825597 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:23:58.827062 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:58.827542 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:58.827543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:58.827544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:58.827599 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:58.827892 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:58.827893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:58.827894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:58.828013 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:23:58.828055 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:58.828542 (+ 487us) server_negotiation.cc:299] Negotiation successful
0620 09:23:58.828626 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:23:58.828936 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:58.816541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:33517 (local address 127.4.59.3:43077)
0620 09:23:58.816684 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:58.816715 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:58.817227 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:58.817553 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:58.817558 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:58.817791 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:23:58.818057 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:58.818065 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.819003 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.819005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:58.819979 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:58.819982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.820539 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.820541 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:58.820687 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:58.821245 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:23:58.821254 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:23:58.825470 (+ 4216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:23:58.827158 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:58.827165 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:58.827167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:58.827450 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:58.827669 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:23:58.827670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:23:58.827670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:23:58.827818 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:23:58.828099 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:23:58.828104 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:23:58.828397 (+ 293us) client_negotiation.cc:769] Sending connection context
0620 09:23:58.828519 (+ 122us) client_negotiation.cc:240] Negotiation successful
0620 09:23:58.828679 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":93,"threads_started":1}
I20260620 09:23:58.829510 17585 heartbeater.cc:347] Connected to a master server at 127.4.59.62:33517
I20260620 09:23:58.829604 17585 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:58.829761 17585 heartbeater.cc:511] Master 127.4.59.62:33517 requested a full tablet report, sending...
W20260620 09:23:58.830731 17585 heartbeater.cc:644] Failed to heartbeat to 127.4.59.62:33517 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 09:23:58.830806 17585 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:58.830871 17585 heartbeater.cc:511] Master 127.4.59.62:33517 requested a full tablet report, sending...
I20260620 09:23:58.831264 17585 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:58.831350 17585 heartbeater.cc:511] Master 127.4.59.62:33517 requested a full tablet report, sending...
W20260620 09:23:58.831686 17585 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 09:23:58.834915 17589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:58.824365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49194 (local address 127.4.59.62:33517)
0620 09:23:58.824514 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:58.824516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:58.824729 (+ 213us) server_negotiation.cc:407] Connection header received
0620 09:23:58.824805 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:58.824806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:58.824818 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:58.824878 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:23:58.825820 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.826544 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.829173 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:58.829460 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:58.829520 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:58.831670 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:23:58.831671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:23:58.831672 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:23:58.831678 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:23:58.833068 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:58.833448 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:58.833449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:58.833449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:58.833508 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:58.834360 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:58.834361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:58.834361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:58.834481 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:23:58.834679 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:58.834716 (+ 37us) server_negotiation.cc:299] Negotiation successful
0620 09:23:58.834810 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":54,"threads_started":1}
Jun 20 09:23:59 dist-test-slave-414w krb5kdc[16123](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781947439, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:23:59.453513 17317 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:59.453893 17317 heartbeater.cc:511] Master 127.4.59.62:33517 requested a full tablet report, sending...
I20260620 09:23:59.454257 17119 ts_manager.cc:194] Registered new tserver with Master: c85d4a8d9dce4297844500a6e48de248 (127.4.59.1:36565)
I20260620 09:23:59.455312 17119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:49079
I20260620 09:23:59.660096 17451 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:59.660264 17451 heartbeater.cc:511] Master 127.4.59.62:33517 requested a full tablet report, sending...
I20260620 09:23:59.660576 17119 ts_manager.cc:194] Registered new tserver with Master: 3b74a34f762e410aac3831fa6b620f57 (127.4.59.2:34789)
I20260620 09:23:59.661120 17119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:38447
Jun 20 09:23:59 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947439, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
I20260620 09:23:59.831889 17585 heartbeater.cc:464] Registering TS with master...
I20260620 09:23:59.832048 17585 heartbeater.cc:511] Master 127.4.59.62:33517 requested a full tablet report, sending...
I20260620 09:23:59.832408 17119 ts_manager.cc:194] Registered new tserver with Master: 599482cac8f1486e8e8221cb147563a6 (127.4.59.3:38527)
I20260620 09:23:59.832944 17119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:43077
I20260620 09:23:59.834384 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:23:59.843614 17606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:59.835439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33700 (local address 127.4.59.62:33517)
0620 09:23:59.835616 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:59.835618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:59.835736 (+ 118us) server_negotiation.cc:407] Connection header received
0620 09:23:59.835837 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:59.835838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:59.835849 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:59.835890 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:23:59.836628 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.837428 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.838518 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.838820 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.838861 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:59.840419 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:23:59.840421 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:23:59.840421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:23:59.840428 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:23:59.842285 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:59.842694 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:59.842695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:59.842696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:59.842774 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:23:59.842996 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:23:59.842997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:23:59.842997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:23:59.843127 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:23:59.843203 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:59.843421 (+ 218us) server_negotiation.cc:299] Negotiation successful
0620 09:23:59.843510 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":67,"threads_started":1}
I20260620 09:23:59.846282 17119 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:23:59.848547 17119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:23:59.861364 17612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:59.851732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:38527 (local address 127.0.0.1:60236)
0620 09:23:59.852691 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:59.852742 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:59.852814 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:59.853312 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:59.853315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:59.853317 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:23:59.853381 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:59.853382 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.856563 (+ 3181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.856564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:59.859834 (+ 3270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:59.859836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.861021 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.861022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:59.861104 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:59.861117 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 09:23:59.861209 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 09:23:59.861264 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":932,"thread_start_us":75,"threads_started":1}
I20260620 09:23:59.861869 17615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:59.851962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60236 (local address 127.4.59.3:38527)
0620 09:23:59.852623 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:59.852627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:59.852895 (+ 268us) server_negotiation.cc:407] Connection header received
0620 09:23:59.852931 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:59.852934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:59.852949 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:59.853022 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:23:59.853468 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.855189 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.859928 (+ 4739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.860698 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.860773 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:59.860836 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:59.861703 (+ 867us) server_negotiation.cc:299] Negotiation successful
0620 09:23:59.861747 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":54,"threads_started":1}
I20260620 09:23:59.863791 17616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:59.852143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37066 (local address 127.4.59.2:34789)
0620 09:23:59.852274 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:59.852280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:59.853220 (+ 940us) server_negotiation.cc:407] Connection header received
0620 09:23:59.854663 (+ 1443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:59.854666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:59.854684 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:59.854751 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:23:59.855447 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.856484 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.862103 (+ 5619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.863281 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.863328 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:59.863376 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:59.863654 (+ 278us) server_negotiation.cc:299] Negotiation successful
0620 09:23:59.863709 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260620 09:23:59.864146 17614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:59.851733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:34789 (local address 127.0.0.1:37066)
0620 09:23:59.853127 (+ 1394us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:59.853163 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:59.853248 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:59.855311 (+ 2063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:59.855312 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:59.855314 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:23:59.855362 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:59.855364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.856924 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.856925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:59.858278 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:59.858281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.863437 (+ 5156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.863437 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:59.863501 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:59.863504 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:23:59.864020 (+ 516us) client_negotiation.cc:240] Negotiation successful
0620 09:23:59.864072 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1376,"thread_start_us":81,"threads_started":1}
I20260620 09:23:59.864897 17518 tablet_service.cc:1467] Processing CreateTablet for tablet dd6d68285e2e477ea547efc2e5226b1e (DEFAULT_TABLE table=test-table [id=218809f1756a4668b52837dea8cc6171]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:23:59.865268 17518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd6d68285e2e477ea547efc2e5226b1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:23:59.869316 17374 tablet_service.cc:1467] Processing CreateTablet for tablet dd6d68285e2e477ea547efc2e5226b1e (DEFAULT_TABLE table=test-table [id=218809f1756a4668b52837dea8cc6171]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:23:59.869614 17374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd6d68285e2e477ea547efc2e5226b1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:23:59.876709 17613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:59.851945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:36565 (local address 127.0.0.1:39104)
0620 09:23:59.860902 (+ 8957us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:59.860914 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:59.860959 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:59.863978 (+ 3019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:59.863979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:59.863981 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:23:59.864072 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:59.864074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.874213 (+ 10139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.874214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:59.875418 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:59.875421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.876457 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.876457 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:59.876523 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:59.876528 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 09:23:59.876574 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 09:23:59.876627 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8937,"thread_start_us":69,"threads_started":1}
I20260620 09:23:59.877185 17617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:59.852220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39104 (local address 127.4.59.1:36565)
0620 09:23:59.863740 (+ 11520us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:59.863746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:59.863757 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:23:59.863793 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:59.863795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:59.863811 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:59.863895 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:23:59.864152 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.874105 (+ 9953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.875524 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.876300 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.876370 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:59.876422 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:59.877043 (+ 621us) server_negotiation.cc:299] Negotiation successful
0620 09:23:59.877085 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11478,"thread_start_us":53,"threads_started":1}
I20260620 09:23:59.877619 17619 tablet_bootstrap.cc:492] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57: Bootstrap starting.
I20260620 09:23:59.878055 17233 tablet_service.cc:1467] Processing CreateTablet for tablet dd6d68285e2e477ea547efc2e5226b1e (DEFAULT_TABLE table=test-table [id=218809f1756a4668b52837dea8cc6171]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:23:59.878285 17619 tablet_bootstrap.cc:654] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57: Neither blocks nor log segments found. Creating new log.
I20260620 09:23:59.878396 17233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd6d68285e2e477ea547efc2e5226b1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:23:59.878750 17618 tablet_bootstrap.cc:492] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6: Bootstrap starting.
I20260620 09:23:59.879376 17618 tablet_bootstrap.cc:654] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6: Neither blocks nor log segments found. Creating new log.
I20260620 09:23:59.879681 17618 log.cc:824] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6: Log is configured to *not* fsync() on all Append() calls
I20260620 09:23:59.880052 17619 log.cc:824] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57: Log is configured to *not* fsync() on all Append() calls
I20260620 09:23:59.880780 17618 tablet_bootstrap.cc:492] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6: No bootstrap required, opened a new log
I20260620 09:23:59.880836 17618 ts_tablet_manager.cc:1397] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:23:59.882906 17618 raft_consensus.cc:348] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b74a34f762e410aac3831fa6b620f57" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34789 } } peers { permanent_uuid: "599482cac8f1486e8e8221cb147563a6" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38527 } } peers { permanent_uuid: "c85d4a8d9dce4297844500a6e48de248" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36565 } }
I20260620 09:23:59.883167 17618 raft_consensus.cc:374] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:23:59.883249 17618 raft_consensus.cc:729] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 599482cac8f1486e8e8221cb147563a6, State: Initialized, Role: FOLLOWER
I20260620 09:23:59.883428 17618 consensus_queue.cc:260] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b74a34f762e410aac3831fa6b620f57" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34789 } } peers { permanent_uuid: "599482cac8f1486e8e8221cb147563a6" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38527 } } peers { permanent_uuid: "c85d4a8d9dce4297844500a6e48de248" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36565 } }
I20260620 09:23:59.883756 17618 ts_tablet_manager.cc:1428] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:23:59.883905 17585 heartbeater.cc:503] Master 127.4.59.62:33517 was elected leader, sending a full tablet report...
I20260620 09:23:59.885252 17619 tablet_bootstrap.cc:492] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57: No bootstrap required, opened a new log
I20260620 09:23:59.885370 17619 ts_tablet_manager.cc:1397] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260620 09:23:59.888001 17619 raft_consensus.cc:348] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b74a34f762e410aac3831fa6b620f57" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34789 } } peers { permanent_uuid: "599482cac8f1486e8e8221cb147563a6" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38527 } } peers { permanent_uuid: "c85d4a8d9dce4297844500a6e48de248" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36565 } }
I20260620 09:23:59.888206 17619 raft_consensus.cc:374] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:23:59.888280 17619 raft_consensus.cc:729] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b74a34f762e410aac3831fa6b620f57, State: Initialized, Role: FOLLOWER
I20260620 09:23:59.888446 17619 consensus_queue.cc:260] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b74a34f762e410aac3831fa6b620f57" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34789 } } peers { permanent_uuid: "599482cac8f1486e8e8221cb147563a6" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38527 } } peers { permanent_uuid: "c85d4a8d9dce4297844500a6e48de248" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36565 } }
I20260620 09:23:59.888736 17619 ts_tablet_manager.cc:1428] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 09:23:59.888875 17451 heartbeater.cc:503] Master 127.4.59.62:33517 was elected leader, sending a full tablet report...
I20260620 09:23:59.890329 17622 tablet_bootstrap.cc:492] T dd6d68285e2e477ea547efc2e5226b1e P c85d4a8d9dce4297844500a6e48de248: Bootstrap starting.
I20260620 09:23:59.890959 17622 tablet_bootstrap.cc:654] T dd6d68285e2e477ea547efc2e5226b1e P c85d4a8d9dce4297844500a6e48de248: Neither blocks nor log segments found. Creating new log.
I20260620 09:23:59.891296 17622 log.cc:824] T dd6d68285e2e477ea547efc2e5226b1e P c85d4a8d9dce4297844500a6e48de248: Log is configured to *not* fsync() on all Append() calls
I20260620 09:23:59.892030 17622 tablet_bootstrap.cc:492] T dd6d68285e2e477ea547efc2e5226b1e P c85d4a8d9dce4297844500a6e48de248: No bootstrap required, opened a new log
I20260620 09:23:59.892089 17622 ts_tablet_manager.cc:1397] T dd6d68285e2e477ea547efc2e5226b1e P c85d4a8d9dce4297844500a6e48de248: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:23:59.894519 17622 raft_consensus.cc:348] T dd6d68285e2e477ea547efc2e5226b1e P c85d4a8d9dce4297844500a6e48de248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b74a34f762e410aac3831fa6b620f57" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34789 } } peers { permanent_uuid: "599482cac8f1486e8e8221cb147563a6" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38527 } } peers { permanent_uuid: "c85d4a8d9dce4297844500a6e48de248" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36565 } }
I20260620 09:23:59.894680 17622 raft_consensus.cc:374] T dd6d68285e2e477ea547efc2e5226b1e P c85d4a8d9dce4297844500a6e48de248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:23:59.894711 17622 raft_consensus.cc:729] T dd6d68285e2e477ea547efc2e5226b1e P c85d4a8d9dce4297844500a6e48de248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c85d4a8d9dce4297844500a6e48de248, State: Initialized, Role: FOLLOWER
I20260620 09:23:59.894819 17622 consensus_queue.cc:260] T dd6d68285e2e477ea547efc2e5226b1e P c85d4a8d9dce4297844500a6e48de248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b74a34f762e410aac3831fa6b620f57" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34789 } } peers { permanent_uuid: "599482cac8f1486e8e8221cb147563a6" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38527 } } peers { permanent_uuid: "c85d4a8d9dce4297844500a6e48de248" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36565 } }
I20260620 09:23:59.895210 17622 ts_tablet_manager.cc:1428] T dd6d68285e2e477ea547efc2e5226b1e P c85d4a8d9dce4297844500a6e48de248: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:23:59.895356 17317 heartbeater.cc:503] Master 127.4.59.62:33517 was elected leader, sending a full tablet report...
W20260620 09:23:59.900925 17452 tablet.cc:2369] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 09:23:59.935482 17318 tablet.cc:2369] T dd6d68285e2e477ea547efc2e5226b1e P c85d4a8d9dce4297844500a6e48de248: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:23:59.970481 17623 raft_consensus.cc:482] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:23:59.970644 17623 raft_consensus.cc:504] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b74a34f762e410aac3831fa6b620f57" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34789 } } peers { permanent_uuid: "599482cac8f1486e8e8221cb147563a6" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38527 } } peers { permanent_uuid: "c85d4a8d9dce4297844500a6e48de248" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36565 } }
I20260620 09:23:59.971055 17623 leader_election.cc:288] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b74a34f762e410aac3831fa6b620f57 (127.4.59.2:34789), c85d4a8d9dce4297844500a6e48de248 (127.4.59.1:36565)
Jun 20 09:23:59 dist-test-slave-414w krb5kdc[16123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947439, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for HTTP/127.4.59.20@KRBTEST.COM
I20260620 09:23:59.977713 17617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:59.971846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:38483 (local address 127.4.59.1:36565)
0620 09:23:59.971891 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:59.971893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:59.972048 (+ 155us) server_negotiation.cc:407] Connection header received
0620 09:23:59.972178 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:59.972179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:59.972191 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:59.972223 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:23:59.972456 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.973294 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.974885 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.975482 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.975533 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:59.975545 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:59.977574 (+ 2029us) server_negotiation.cc:299] Negotiation successful
0620 09:23:59.977633 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 09:23:59.978420 17629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:59.971330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:34789 (local address 127.4.59.3:57989)
0620 09:23:59.971856 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:59.971867 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:59.971953 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:59.972136 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:59.972137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:59.972139 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:23:59.972214 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:59.972216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.976020 (+ 3804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.976021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:59.977230 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:59.977234 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.978132 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.978133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:59.978221 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:59.978225 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:23:59.978288 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 09:23:59.978329 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":105,"threads_started":1}
I20260620 09:23:59.978425 17616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:59.971782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:57989 (local address 127.4.59.2:34789)
0620 09:23:59.971847 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:59.971849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:59.971923 (+ 74us) server_negotiation.cc:407] Connection header received
0620 09:23:59.972027 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:59.972028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:59.972041 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:59.972070 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:23:59.972283 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.972994 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.977350 (+ 4356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.977991 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.978041 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:59.978051 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:23:59.978307 (+ 256us) server_negotiation.cc:299] Negotiation successful
0620 09:23:59.978353 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 09:23:59.978969 17405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6d68285e2e477ea547efc2e5226b1e" candidate_uuid: "599482cac8f1486e8e8221cb147563a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b74a34f762e410aac3831fa6b620f57" is_pre_election: true
I20260620 09:23:59.979138 17405 raft_consensus.cc:2454] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 599482cac8f1486e8e8221cb147563a6 in term 0.
I20260620 09:23:59.979367 17474 leader_election.cc:302] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b74a34f762e410aac3831fa6b620f57, 599482cac8f1486e8e8221cb147563a6; no voters:
I20260620 09:23:59.979489 17623 raft_consensus.cc:2781] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:23:59.979575 17623 raft_consensus.cc:482] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:23:59.979617 17623 raft_consensus.cc:3037] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:23:59.979926 17628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:59.971545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:36565 (local address 127.4.59.3:38483)
0620 09:23:59.971951 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0620 09:23:59.971962 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:23:59.972092 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:23:59.972294 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:23:59.972295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:23:59.972297 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:23:59.972370 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:59.972372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.973397 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.973398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:59.974771 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:23:59.974774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.975627 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.975628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:23:59.975707 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:59.975712 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 09:23:59.979797 (+ 4085us) client_negotiation.cc:240] Negotiation successful
0620 09:23:59.979846 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":76,"threads_started":1}
I20260620 09:23:59.980635 17623 raft_consensus.cc:504] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b74a34f762e410aac3831fa6b620f57" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34789 } } peers { permanent_uuid: "599482cac8f1486e8e8221cb147563a6" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38527 } } peers { permanent_uuid: "c85d4a8d9dce4297844500a6e48de248" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36565 } }
I20260620 09:23:59.980785 17623 leader_election.cc:288] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [CANDIDATE]: Term 1 election: Requested vote from peers 3b74a34f762e410aac3831fa6b620f57 (127.4.59.2:34789), c85d4a8d9dce4297844500a6e48de248 (127.4.59.1:36565)
I20260620 09:23:59.981055 17255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6d68285e2e477ea547efc2e5226b1e" candidate_uuid: "599482cac8f1486e8e8221cb147563a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c85d4a8d9dce4297844500a6e48de248" is_pre_election: true
I20260620 09:23:59.981122 17254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6d68285e2e477ea547efc2e5226b1e" candidate_uuid: "599482cac8f1486e8e8221cb147563a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c85d4a8d9dce4297844500a6e48de248"
I20260620 09:23:59.981220 17255 raft_consensus.cc:2454] T dd6d68285e2e477ea547efc2e5226b1e P c85d4a8d9dce4297844500a6e48de248 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 599482cac8f1486e8e8221cb147563a6 in term 0.
I20260620 09:23:59.981513 17405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6d68285e2e477ea547efc2e5226b1e" candidate_uuid: "599482cac8f1486e8e8221cb147563a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b74a34f762e410aac3831fa6b620f57"
I20260620 09:23:59.981590 17405 raft_consensus.cc:3037] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:23:59.982431 17405 raft_consensus.cc:2454] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 599482cac8f1486e8e8221cb147563a6 in term 1.
I20260620 09:23:59.982637 17474 leader_election.cc:302] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b74a34f762e410aac3831fa6b620f57, 599482cac8f1486e8e8221cb147563a6; no voters: c85d4a8d9dce4297844500a6e48de248
I20260620 09:23:59.982793 17623 raft_consensus.cc:2781] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:23:59.982970 17623 raft_consensus.cc:686] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [term 1 LEADER]: Becoming Leader. State: Replica: 599482cac8f1486e8e8221cb147563a6, State: Running, Role: LEADER
I20260620 09:23:59.983064 17623 consensus_queue.cc:237] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b74a34f762e410aac3831fa6b620f57" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34789 } } peers { permanent_uuid: "599482cac8f1486e8e8221cb147563a6" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38527 } } peers { permanent_uuid: "c85d4a8d9dce4297844500a6e48de248" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36565 } }
I20260620 09:23:59.983917 17119 catalog_manager.cc:5495] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 599482cac8f1486e8e8221cb147563a6 (127.4.59.3). New cstate: current_term: 1 leader_uuid: "599482cac8f1486e8e8221cb147563a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b74a34f762e410aac3831fa6b620f57" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "599482cac8f1486e8e8221cb147563a6" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c85d4a8d9dce4297844500a6e48de248" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260620 09:24:00.000126 17615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:23:59.996746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60246 (local address 127.4.59.3:38527)
0620 09:23:59.996795 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 09:23:59.996797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:23:59.996806 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:23:59.996882 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:23:59.996883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:23:59.996920 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:23:59.996981 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:23:59.997196 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.997957 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.999091 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:23:59.999374 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:23:59.999432 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:23:59.999616 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:23:59.999753 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:23:59.999821 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:00.000027 (+ 206us) server_negotiation.cc:299] Negotiation successful
0620 09:24:00.000048 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 09:24:00.002245 17405 raft_consensus.cc:1261] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57 [term 1 FOLLOWER]: Refusing update from remote peer 599482cac8f1486e8e8221cb147563a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:24:00.002218 17254 raft_consensus.cc:3037] T dd6d68285e2e477ea547efc2e5226b1e P c85d4a8d9dce4297844500a6e48de248 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:00.002599 17630 consensus_queue.cc:1035] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b74a34f762e410aac3831fa6b620f57" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:24:00.003252 17254 raft_consensus.cc:1261] T dd6d68285e2e477ea547efc2e5226b1e P c85d4a8d9dce4297844500a6e48de248 [term 1 FOLLOWER]: Refusing update from remote peer 599482cac8f1486e8e8221cb147563a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:24:00.003713 17630 consensus_queue.cc:1035] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c85d4a8d9dce4297844500a6e48de248" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:24:00.006314 17633 mvcc.cc:205] Tried to move back new op lower bound from 7298856714247561216 to 7298856714172547072. Current Snapshot: MvccSnapshot[applied={T|T < 7298856714247561216}]
I20260620 09:24:00.008898 17119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:24:00.008997 17119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:24:00.011119 17119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f73066edab2b42f0b8d3ca4fd3aefa34: Sending DeleteTablet for 3 replicas of tablet dd6d68285e2e477ea547efc2e5226b1e
I20260620 09:24:00.011704 17233 tablet_service.cc:1514] Processing DeleteTablet for tablet dd6d68285e2e477ea547efc2e5226b1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:00 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:39104
I20260620 09:24:00.011983 17374 tablet_service.cc:1514] Processing DeleteTablet for tablet dd6d68285e2e477ea547efc2e5226b1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:00 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:37066
I20260620 09:24:00.012331 17518 tablet_service.cc:1514] Processing DeleteTablet for tablet dd6d68285e2e477ea547efc2e5226b1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:00 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:60236
I20260620 09:24:00.013350 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 17158
I20260620 09:24:00.017668 17634 mvcc.cc:205] Tried to move back new op lower bound from 7298856714247561216 to 7298856714172547072. Current Snapshot: MvccSnapshot[applied={T|T < 7298856714247561216}]
I20260620 09:24:00.021481 17642 tablet_replica.cc:330] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57: stopping tablet replica
I20260620 09:24:00.021703 17642 raft_consensus.cc:2229] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:24:00.021857 17642 raft_consensus.cc:2258] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:24:00.022037 17643 tablet_replica.cc:330] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6: stopping tablet replica
I20260620 09:24:00.022289 17643 raft_consensus.cc:2229] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:24:00.022454 17643 raft_consensus.cc:2258] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:24:00.023082 17643 ts_tablet_manager.cc:1905] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:24:00.024935 17643 ts_tablet_manager.cc:1918] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:24:00.026139 17643 log.cc:1196] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/dd6d68285e2e477ea547efc2e5226b1e
I20260620 09:24:00.026242 17642 ts_tablet_manager.cc:1905] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:24:00.026518 17643 ts_tablet_manager.cc:1939] T dd6d68285e2e477ea547efc2e5226b1e P 599482cac8f1486e8e8221cb147563a6: Deleting consensus metadata
I20260620 09:24:00.028357 17642 ts_tablet_manager.cc:1918] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:24:00.028496 17642 log.cc:1196] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/dd6d68285e2e477ea547efc2e5226b1e
I20260620 09:24:00.028841 17642 ts_tablet_manager.cc:1939] T dd6d68285e2e477ea547efc2e5226b1e P 3b74a34f762e410aac3831fa6b620f57: Deleting consensus metadata
I20260620 09:24:00.029398 17105 catalog_manager.cc:4841] TS 3b74a34f762e410aac3831fa6b620f57 (127.4.59.2:34789): tablet dd6d68285e2e477ea547efc2e5226b1e (table test-table [id=218809f1756a4668b52837dea8cc6171]) successfully deleted
W20260620 09:24:00.033614 17105 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.59.1:36565 (error 108)
I20260620 09:24:00.033911 17104 catalog_manager.cc:4841] TS 599482cac8f1486e8e8221cb147563a6 (127.4.59.3:38527): tablet dd6d68285e2e477ea547efc2e5226b1e (table test-table [id=218809f1756a4668b52837dea8cc6171]) successfully deleted
I20260620 09:24:00.034322 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 17321
I20260620 09:24:00.035661 17613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:00.035534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:36565 (local address 127.0.0.1:39120)
0620 09:24:00.035588 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:00.035623 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:36565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 09:24:00.035784 17105 catalog_manager.cc:4584] TS c85d4a8d9dce4297844500a6e48de248 (127.4.59.1:36565): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dd6d68285e2e477ea547efc2e5226b1e: Network error: Client connection negotiation failed: client connection to 127.4.59.1:36565: connect: Connection refused (error 111)
I20260620 09:24:00.043087 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 17454
I20260620 09:24:00.051021 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 17086
I20260620 09:24:00.060467 4332 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 09:24:00.404754 4332 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 09:24:00.404847 4332 mini_ranger.cc:66] Stopping Ranger...
I20260620 09:24:00.748883 4332 mini_ranger.cc:68] Stopped Ranger
2026-06-20 09:24:00.749 UTC [16153] LOG: received smart shutdown request
2026-06-20 09:24:00.753 UTC [16153] LOG: background worker "logical replication launcher" (PID 16162) exited with exit code 1
2026-06-20 09:24:00.753 UTC [16157] LOG: shutting down
2026-06-20 09:24:00.912 UTC [16153] LOG: database system is shut down
2026-06-20T09:24:00Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47291 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:24:01 dist-test-slave-414w krb5kdc[17655](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:24:01 dist-test-slave-414w krb5kdc[17655](info): set up 2 sockets
Jun 20 09:24:01 dist-test-slave-414w krb5kdc[17655](info): commencing operation
krb5kdc: starting...
W20260620 09:24:02.057309 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:24:02 dist-test-slave-414w krb5kdc[17655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:24:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:24:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:24:02.177778 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:40839
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:33683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:40839,127.4.59.61:35361,127.4.59.60:39605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:24:02.256525 17668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:02.256704 17668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:02.256726 17668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:02.257800 17668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:24:02.257834 17668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:02.257848 17668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:02.257861 17668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:24:02.257874 17668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:24:02.259266 17668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:33683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:40839,127.4.59.61:35361,127.4.59.60:39605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:40839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:02.259514 17668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:02.259720 17668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:02.264536 17674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:02.264577 17668 server_base.cc:1034] running on GCE node
W20260620 09:24:02.264524 17676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:02.264524 17673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:02.264968 17668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:02.265285 17668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:02.266431 17668 hybrid_clock.cc:648] HybridClock initialized: now 1781947442266423 us; error 39 us; skew 500 ppm
Jun 20 09:24:02 dist-test-slave-414w krb5kdc[17655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:02.272310 17668 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:24:02.272755 17668 webserver.cc:460] Webserver started at http://127.4.59.62:33405/ using document root <none> and password file <none>
I20260620 09:24:02.272989 17668 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:02.273061 17668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:02.273231 17668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:24:02.274228 17668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "00994bc94ca947c9af3e68093a8c03d0"
format_stamp: "Formatted at 2026-06-20 09:24:02 on dist-test-slave-414w"
server_key: "63fa603bea28b0502de5702ca92ee6cf"
server_key_iv: "55c55772be8c570941e645aa20a67465"
server_key_version: "encryptionkey@0"
I20260620 09:24:02.274588 17668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "00994bc94ca947c9af3e68093a8c03d0"
format_stamp: "Formatted at 2026-06-20 09:24:02 on dist-test-slave-414w"
server_key: "63fa603bea28b0502de5702ca92ee6cf"
server_key_iv: "55c55772be8c570941e645aa20a67465"
server_key_version: "encryptionkey@0"
I20260620 09:24:02.275888 17668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 09:24:02.276659 17683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:02.276862 17668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:24:02.276954 17668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "00994bc94ca947c9af3e68093a8c03d0"
format_stamp: "Formatted at 2026-06-20 09:24:02 on dist-test-slave-414w"
server_key: "63fa603bea28b0502de5702ca92ee6cf"
server_key_iv: "55c55772be8c570941e645aa20a67465"
server_key_version: "encryptionkey@0"
I20260620 09:24:02.277045 17668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:02.293931 17668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:02.296085 17668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:02.296239 17668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:02.300185 17668 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:40839
I20260620 09:24:02.300215 17735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:40839 every 8 connection(s)
I20260620 09:24:02.300534 17668 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:24:02.301386 17736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:02.302428 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 17668
I20260620 09:24:02.302557 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:24:02.302733 4332 external_mini_cluster.cc:1428] Setting key 49d04a11c0029a7a07cf5a068304cce5
I20260620 09:24:02.303020 17736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 09:24:02 dist-test-slave-414w krb5kdc[17655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.59.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.61@KRBTEST.COM" created.
I20260620 09:24:02.312424 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.303655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40839 (local address 127.0.0.1:32900)
0620 09:24:02.303808 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.303849 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:02.304337 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:02.304629 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:02.304632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:02.304894 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:02.305034 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.305038 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.306060 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.306061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.306716 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.306718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.307217 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.307218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.307279 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.307785 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:02.307793 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:02.309093 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:02.311040 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.311046 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.311048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.311229 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.311477 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.311478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.311478 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.311586 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.311911 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:02.311915 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:02.311968 (+ 53us) client_negotiation.cc:769] Sending connection context
0620 09:24:02.312036 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 09:24:02.312171 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260620 09:24:02.312428 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.303840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32900 (local address 127.4.59.62:40839)
0620 09:24:02.303970 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:02.303974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:02.304001 (+ 27us) server_negotiation.cc:407] Connection header received
0620 09:24:02.304500 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:02.304502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:02.304518 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:02.304620 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:02.305166 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.305948 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.306814 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.307121 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.307245 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.309204 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:02.309211 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:02.309212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:02.309218 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:02.310964 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.311303 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.311305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.311306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.311404 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.311655 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.311656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.311656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.311842 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:02.311906 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:02.312098 (+ 192us) server_negotiation.cc:299] Negotiation successful
0620 09:24:02.312231 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":108,"thread_start_us":55,"threads_started":1}
I20260620 09:24:02.313186 17736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } has no permanent_uuid. Determining permanent_uuid...
I20260620 09:24:02.313521 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.313410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:35361 (local address 127.0.0.1:56920)
0620 09:24:02.313457 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.313504 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:35361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 09:24:02.313695 17686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.59.61:35361: connect: Connection refused (error 111)
I20260620 09:24:02.314067 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.313971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:35361 (local address 127.0.0.1:56934)
0620 09:24:02.314022 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.314045 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:35361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260620 09:24:02.314199 17736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.61:35361: Network error: Client connection negotiation failed: client connection to 127.4.59.61:35361: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.61.keytab.
Entry for principal kudu/127.4.59.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.61.keytab.
I20260620 09:24:02.327909 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.61:35361
--webserver_interface=127.4.59.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.61.keytab
--principal=kudu/127.4.59.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:33683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:40839,127.4.59.61:35361,127.4.59.60:39605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:24:02.363451 17736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } attempt: 1
I20260620 09:24:02.363899 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.363763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:35361 (local address 127.0.0.1:56938)
0620 09:24:02.363826 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.363869 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:35361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260620 09:24:02.364262 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.364167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:35361 (local address 127.0.0.1:56950)
0620 09:24:02.364212 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.364238 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:35361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 09:24:02.364388 17736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.61:35361: Network error: Client connection negotiation failed: client connection to 127.4.59.61:35361: connect: Connection refused (error 111)
W20260620 09:24:02.406841 17743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:02.407070 17743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:02.407105 17743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:02.408216 17743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:24:02.408283 17743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:02.408308 17743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:02.408335 17743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:24:02.408365 17743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:24:02.409895 17743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:33683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:40839,127.4.59.61:35361,127.4.59.60:39605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.61.keytab
--principal=kudu/127.4.59.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.61:35361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.59.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:02.410156 17743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:02.410367 17743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:02.414938 17749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:02.414988 17751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:02.414949 17748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:02.415102 17743 server_base.cc:1034] running on GCE node
I20260620 09:24:02.415421 17743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:02.415745 17743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:02.416908 17743 hybrid_clock.cc:648] HybridClock initialized: now 1781947442416904 us; error 34 us; skew 500 ppm
Jun 20 09:24:02 dist-test-slave-414w krb5kdc[17655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:02.419461 17743 init.cc:377] Logged in from keytab as kudu/127.4.59.61@KRBTEST.COM (short username kudu)
I20260620 09:24:02.419850 17743 webserver.cc:460] Webserver started at http://127.4.59.61:41017/ using document root <none> and password file <none>
I20260620 09:24:02.420073 17743 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:02.420138 17743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:02.420269 17743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:24:02.421358 17743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/data/instance:
uuid: "710467195dbd4df8a756e2c0283474f4"
format_stamp: "Formatted at 2026-06-20 09:24:02 on dist-test-slave-414w"
server_key: "dff6de26fde18c9f8de4d25ad3fd110f"
server_key_iv: "b21e8862e42723e21af3dbbbf50641c2"
server_key_version: "encryptionkey@0"
I20260620 09:24:02.421738 17743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/wal/instance:
uuid: "710467195dbd4df8a756e2c0283474f4"
format_stamp: "Formatted at 2026-06-20 09:24:02 on dist-test-slave-414w"
server_key: "dff6de26fde18c9f8de4d25ad3fd110f"
server_key_iv: "b21e8862e42723e21af3dbbbf50641c2"
server_key_version: "encryptionkey@0"
I20260620 09:24:02.423017 17743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:02.423730 17758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:02.424007 17743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:02.424108 17743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/wal
uuid: "710467195dbd4df8a756e2c0283474f4"
format_stamp: "Formatted at 2026-06-20 09:24:02 on dist-test-slave-414w"
server_key: "dff6de26fde18c9f8de4d25ad3fd110f"
server_key_iv: "b21e8862e42723e21af3dbbbf50641c2"
server_key_version: "encryptionkey@0"
I20260620 09:24:02.424177 17743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:02.432533 17736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } attempt: 2
I20260620 09:24:02.432931 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.432789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:35361 (local address 127.0.0.1:56960)
0620 09:24:02.432843 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.432896 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:35361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 09:24:02.433364 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.433260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:35361 (local address 127.0.0.1:56974)
0620 09:24:02.433306 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.433333 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:35361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 09:24:02.433445 17736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.61:35361: Network error: Client connection negotiation failed: client connection to 127.4.59.61:35361: connect: Connection refused (error 111)
I20260620 09:24:02.444365 17743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:02.446615 17743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:02.446767 17743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:02.450587 17743 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.61:35361
I20260620 09:24:02.450619 17810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.61:35361 every 8 connection(s)
I20260620 09:24:02.450977 17743 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/data/info.pb
I20260620 09:24:02.451787 17811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:02.452147 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 17743
I20260620 09:24:02.452263 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-1/wal/instance
I20260620 09:24:02.452457 4332 external_mini_cluster.cc:1428] Setting key f5dcf40cd7cba6b5a7cef870f9d73b25
I20260620 09:24:02.453429 17811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 09:24:02 dist-test-slave-414w krb5kdc[17655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.61@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.59.60@KRBTEST.COM; defaulting to no policy
I20260620 09:24:02.462275 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.454118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32914 (local address 127.4.59.62:40839)
0620 09:24:02.454184 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:02.454187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:02.454346 (+ 159us) server_negotiation.cc:407] Connection header received
0620 09:24:02.454751 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:02.454753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:02.454768 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:02.454835 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:02.455396 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.456208 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.457228 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.457583 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.457664 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.459636 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:02.459638 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:02.459638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:02.459646 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:02.461066 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.461464 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.461465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.461466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.461517 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.461719 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.461720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.461720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.461851 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:02.461942 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:02.462150 (+ 208us) server_negotiation.cc:299] Negotiation successful
0620 09:24:02.462213 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 09:24:02.462457 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.454056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40839 (local address 127.0.0.1:32914)
0620 09:24:02.454217 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.454262 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:02.454676 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:02.454891 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:02.454893 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:02.455133 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:02.455307 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.455311 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.456316 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.456317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.457091 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.457092 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.457676 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.457677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.457779 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.458257 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:02.458264 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:02.459548 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:02.461135 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.461175 (+ 40us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.461177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.461396 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.461575 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.461576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.461576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.461666 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.461926 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:02.461931 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:02.462040 (+ 109us) client_negotiation.cc:769] Sending connection context
0620 09:24:02.462122 (+ 82us) client_negotiation.cc:240] Negotiation successful
0620 09:24:02.462230 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.60@KRBTEST.COM" created.
I20260620 09:24:02.462994 17811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 09:24:02 dist-test-slave-414w krb5kdc[17655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.61@KRBTEST.COM for kudu/127.4.59.61@KRBTEST.COM
I20260620 09:24:02.470705 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.463227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:35361 (local address 127.0.0.1:56976)
0620 09:24:02.463272 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.463283 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:02.463322 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:02.463599 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:02.463600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:02.463708 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:02.463751 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.463752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.464680 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.464681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.465366 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.465368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.466021 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.466021 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.466073 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.466471 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:02.466475 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:02.467817 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:02.469770 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.469771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.469771 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.469929 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.470147 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.470148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.470148 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.470194 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.470482 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:02.470482 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:02.470514 (+ 32us) client_negotiation.cc:769] Sending connection context
0620 09:24:02.470564 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 09:24:02.470635 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260620 09:24:02.470841 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.463281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56976 (local address 127.4.59.61:35361)
0620 09:24:02.463445 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:02.463450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:02.463473 (+ 23us) server_negotiation.cc:407] Connection header received
0620 09:24:02.463495 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:02.463497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:02.463511 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:02.463588 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:02.463820 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.464600 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.465446 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.465937 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.466019 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.467901 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:02.467910 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:02.467911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:02.467918 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:02.469691 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.470012 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.470015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.470016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.470077 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.470250 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.470251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.470251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.470407 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:02.470505 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:02.470600 (+ 95us) server_negotiation.cc:299] Negotiation successful
0620 09:24:02.470717 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260620 09:24:02.471611 17811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } has no permanent_uuid. Determining permanent_uuid...
I20260620 09:24:02.471923 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.471823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39605 (local address 127.0.0.1:44618)
0620 09:24:02.471868 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.471907 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 09:24:02.472085 17760 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39605: connect: Connection refused (error 111)
I20260620 09:24:02.472514 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.472433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39605 (local address 127.0.0.1:44622)
0620 09:24:02.472476 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.472499 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 09:24:02.472635 17811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.60:39605: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39605: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.60.keytab.
Entry for principal kudu/127.4.59.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.60.keytab.
I20260620 09:24:02.480285 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.60:39605
--webserver_interface=127.4.59.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.60.keytab
--principal=kudu/127.4.59.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:33683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:40839,127.4.59.61:35361,127.4.59.60:39605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:24:02.521862 17811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } attempt: 1
I20260620 09:24:02.522365 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.522225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39605 (local address 127.0.0.1:44624)
0620 09:24:02.522291 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.522336 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 09:24:02.522755 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.522667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39605 (local address 127.0.0.1:44626)
0620 09:24:02.522713 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.522738 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 09:24:02.522859 17811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.60:39605: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39605: connect: Connection refused (error 111)
I20260620 09:24:02.524580 17736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } attempt: 3
Jun 20 09:24:02 dist-test-slave-414w krb5kdc[17655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for kudu/127.4.59.61@KRBTEST.COM
I20260620 09:24:02.531571 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.524924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56986 (local address 127.4.59.61:35361)
0620 09:24:02.524983 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:02.524985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:02.524995 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:24:02.525085 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:02.525086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:02.525096 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:02.525172 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:02.525498 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.526019 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.526824 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.527039 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.527111 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.528993 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:02.528994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:02.528995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:02.529002 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:02.530428 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.530796 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.530798 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.530798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.530859 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.531060 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.531061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.531061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.531171 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:02.531247 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:02.531406 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 09:24:02.531482 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 09:24:02.531576 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.524877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:35361 (local address 127.0.0.1:56986)
0620 09:24:02.524921 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.524939 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:02.525009 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:02.525166 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:02.525167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:02.525334 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:02.525397 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.525400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.526113 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.526115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.526727 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.526730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.527124 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.527125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.527204 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.527684 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:02.527694 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:02.528905 (+ 1211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:02.530531 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.530532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.530532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.530705 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.530933 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.530934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.530934 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.530997 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.531239 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:02.531240 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:02.531295 (+ 55us) client_negotiation.cc:769] Sending connection context
0620 09:24:02.531383 (+ 88us) client_negotiation.cc:240] Negotiation successful
0620 09:24:02.531482 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260620 09:24:02.531903 17736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } has no permanent_uuid. Determining permanent_uuid...
I20260620 09:24:02.532236 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.532139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39605 (local address 127.0.0.1:44630)
0620 09:24:02.532187 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.532213 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 09:24:02.532627 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.532572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39605 (local address 127.0.0.1:44632)
0620 09:24:02.532595 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.532615 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260620 09:24:02.532724 17736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.60:39605: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39605: connect: Connection refused (error 111)
W20260620 09:24:02.560508 17818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:02.560699 17818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:02.560721 17818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:02.561812 17818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:24:02.561846 17818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:02.561861 17818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:02.561874 17818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:24:02.561892 17818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:24:02.563277 17818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:33683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:40839,127.4.59.61:35361,127.4.59.60:39605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.60.keytab
--principal=kudu/127.4.59.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.60:39605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.59.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:02.563453 17818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:02.563609 17818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 09:24:02.563855 17736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } attempt: 1
I20260620 09:24:02.564287 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.564160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39605 (local address 127.0.0.1:44636)
0620 09:24:02.564233 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.564262 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260620 09:24:02.564700 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.564586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39605 (local address 127.0.0.1:44650)
0620 09:24:02.564629 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.564678 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 09:24:02.564795 17736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.60:39605: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39605: connect: Connection refused (error 111)
I20260620 09:24:02.568650 17818 server_base.cc:1034] running on GCE node
W20260620 09:24:02.568755 17826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:02.568755 17823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:02.568874 17824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:02.569214 17818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:02.569525 17818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:02.570694 17818 hybrid_clock.cc:648] HybridClock initialized: now 1781947442570669 us; error 35 us; skew 500 ppm
Jun 20 09:24:02 dist-test-slave-414w krb5kdc[17655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:02.573105 17818 init.cc:377] Logged in from keytab as kudu/127.4.59.60@KRBTEST.COM (short username kudu)
I20260620 09:24:02.573582 17818 webserver.cc:460] Webserver started at http://127.4.59.60:32919/ using document root <none> and password file <none>
I20260620 09:24:02.573800 17818 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:02.573868 17818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:02.574019 17818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:24:02.575062 17818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/data/instance:
uuid: "661d2e6451044ff38adab6b638e43a37"
format_stamp: "Formatted at 2026-06-20 09:24:02 on dist-test-slave-414w"
server_key: "521dd28eca5fb207674fdd64d929ef6a"
server_key_iv: "574b4bc76cf07470179a1f52322ceffb"
server_key_version: "encryptionkey@0"
I20260620 09:24:02.575433 17818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/wal/instance:
uuid: "661d2e6451044ff38adab6b638e43a37"
format_stamp: "Formatted at 2026-06-20 09:24:02 on dist-test-slave-414w"
server_key: "521dd28eca5fb207674fdd64d929ef6a"
server_key_iv: "574b4bc76cf07470179a1f52322ceffb"
server_key_version: "encryptionkey@0"
I20260620 09:24:02.576771 17818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 09:24:02.577529 17833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:02.577807 17818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 09:24:02.577919 17818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/wal
uuid: "661d2e6451044ff38adab6b638e43a37"
format_stamp: "Formatted at 2026-06-20 09:24:02 on dist-test-slave-414w"
server_key: "521dd28eca5fb207674fdd64d929ef6a"
server_key_iv: "574b4bc76cf07470179a1f52322ceffb"
server_key_version: "encryptionkey@0"
I20260620 09:24:02.577991 17818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:02.590996 17811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } attempt: 2
I20260620 09:24:02.591213 17818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:02.591460 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.591305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39605 (local address 127.0.0.1:44658)
0620 09:24:02.591376 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.591411 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 09:24:02.591804 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.591682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39605 (local address 127.0.0.1:44674)
0620 09:24:02.591738 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.591778 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 09:24:02.591922 17811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.60:39605: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39605: connect: Connection refused (error 111)
I20260620 09:24:02.593859 17818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:02.593992 17818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:02.598157 17818 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.60:39605
I20260620 09:24:02.598187 17885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.60:39605 every 8 connection(s)
I20260620 09:24:02.598524 17818 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/data/info.pb
I20260620 09:24:02.599361 17886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:02.600900 17886 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } has no permanent_uuid. Determining permanent_uuid...
I20260620 09:24:02.605713 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 17818
I20260620 09:24:02.605831 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781947262065055-4332-0/minicluster-data/master-2/wal/instance
I20260620 09:24:02.606046 4332 external_mini_cluster.cc:1428] Setting key 7837f8a4e075982d4d65f74ef303c540
Jun 20 09:24:02 dist-test-slave-414w krb5kdc[17655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.60@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:24:02.610213 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.601627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32916 (local address 127.4.59.62:40839)
0620 09:24:02.601707 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:02.601709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:02.601868 (+ 159us) server_negotiation.cc:407] Connection header received
0620 09:24:02.602352 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:02.602353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:02.602365 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:02.602411 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:02.603019 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.603491 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.604528 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.604727 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.604816 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.607425 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:02.607427 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:02.607427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:02.607435 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:02.608710 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.609116 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.609117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.609118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.609195 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.609469 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.609470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.609471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.609593 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:02.609667 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:02.610010 (+ 343us) server_negotiation.cc:299] Negotiation successful
0620 09:24:02.610101 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 09:24:02.610383 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.601573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40839 (local address 127.0.0.1:32916)
0620 09:24:02.601739 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.601784 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:02.602248 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:02.602498 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:02.602501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:02.602731 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:02.602914 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.602918 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.603585 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.603586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.604439 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.604441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.604824 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.604825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.604995 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.605564 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:02.605575 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:02.607329 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:02.608798 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.608805 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.608807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.609024 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.609289 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.609290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.609290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.609401 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.609671 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:02.609677 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:02.609854 (+ 177us) client_negotiation.cc:769] Sending connection context
0620 09:24:02.609964 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 09:24:02.610122 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
I20260620 09:24:02.611011 17886 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 09:24:02 dist-test-slave-414w krb5kdc[17655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.60@KRBTEST.COM
I20260620 09:24:02.615199 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.606460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44684 (local address 127.4.59.60:39605)
0620 09:24:02.606627 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:02.606633 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:02.606679 (+ 46us) server_negotiation.cc:407] Connection header received
0620 09:24:02.606732 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:02.606734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:02.606757 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:02.606845 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:02.607447 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.608446 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.609170 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.609486 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.609577 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.612207 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:02.612213 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:02.612214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:02.612219 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:02.613866 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.614249 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.614251 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.614252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.614307 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.614632 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.614632 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.614633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.614815 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:02.614962 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:02.615007 (+ 45us) server_negotiation.cc:299] Negotiation successful
0620 09:24:02.615089 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
Jun 20 09:24:02 dist-test-slave-414w krb5kdc[17655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.60@KRBTEST.COM for kudu/127.4.59.61@KRBTEST.COM
I20260620 09:24:02.618690 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.611256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:35361 (local address 127.0.0.1:56998)
0620 09:24:02.611745 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.611755 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:02.611801 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:02.611934 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:02.611934 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:02.612039 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:02.612085 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.612086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.612863 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.612864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.613674 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.613676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.614078 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.614079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.614130 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.614614 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:02.614618 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:02.616060 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:02.617762 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.617763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.617763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.617928 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.618123 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.618123 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.618123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.618171 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.618445 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:02.618446 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:02.618488 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 09:24:02.618560 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 09:24:02.618634 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465}
I20260620 09:24:02.618786 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.611364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56998 (local address 127.4.59.61:35361)
0620 09:24:02.611422 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:02.611424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:02.611801 (+ 377us) server_negotiation.cc:407] Connection header received
0620 09:24:02.611838 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:02.611840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:02.611851 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:02.611933 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:02.612156 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.612789 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.613765 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.613999 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.614073 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.616134 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:02.616135 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:02.616136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:02.616143 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:02.617684 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.617997 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.617998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.617999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.618058 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.618263 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.618264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.618264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.618372 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:02.618422 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:02.618593 (+ 171us) server_negotiation.cc:299] Negotiation successful
0620 09:24:02.618669 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 09:24:02.619184 17886 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 09:24:02 dist-test-slave-414w krb5kdc[17655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
Jun 20 09:24:02 dist-test-slave-414w krb5kdc[17655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.60@KRBTEST.COM for kudu/127.4.59.60@KRBTEST.COM
I20260620 09:24:02.623765 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.615936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32930 (local address 127.4.59.62:40839)
0620 09:24:02.615971 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:02.615973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:02.615981 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:24:02.616046 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:02.616047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:02.616056 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:02.616110 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:02.616709 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.617307 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.618235 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.618463 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.618529 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.620894 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:02.620895 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:02.620895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:02.620902 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:02.622644 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.622985 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.622986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.622986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.623040 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.623251 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.623252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.623252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.623354 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:02.623525 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:02.623586 (+ 61us) server_negotiation.cc:299] Negotiation successful
0620 09:24:02.623655 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:24:02.626389 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.619431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39605 (local address 127.0.0.1:44690)
0620 09:24:02.619477 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.619487 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:02.619521 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:02.619644 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:02.619645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:02.619793 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:02.619823 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.619825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.620431 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.620432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.620938 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.620939 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.621453 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.621453 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.621504 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.621939 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:02.621943 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:02.623360 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:02.625433 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.625434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.625434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.625591 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.625802 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.625803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.625803 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.625854 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.626089 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:02.626090 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:02.626125 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 09:24:02.626192 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 09:24:02.626277 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 09:24:02.626441 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.619483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44690 (local address 127.4.59.60:39605)
0620 09:24:02.619518 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:02.619520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:02.619527 (+ 7us) server_negotiation.cc:407] Connection header received
0620 09:24:02.619586 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:02.619587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:02.619594 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:02.619638 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:02.619887 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.620356 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.621030 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.621363 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.621451 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.623499 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:02.623500 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:02.623501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:02.623508 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:02.625342 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.625673 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.625674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.625675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.625722 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.625920 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.625921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.625922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.626024 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:02.626098 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:02.626270 (+ 172us) server_negotiation.cc:299] Negotiation successful
0620 09:24:02.626328 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:24:02.627833 17886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37: Bootstrap starting.
Jun 20 09:24:02 dist-test-slave-414w krb5kdc[17655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.61@KRBTEST.COM
I20260620 09:24:02.628685 17886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:02.629016 17886 log.cc:824] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:02.629868 17886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37: No bootstrap required, opened a new log
I20260620 09:24:02.631253 17886 raft_consensus.cc:348] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } }
I20260620 09:24:02.631413 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.624441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57010 (local address 127.4.59.61:35361)
0620 09:24:02.624477 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:02.624479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:02.624487 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:24:02.624509 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:02.624510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:02.624518 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:02.624572 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:02.625110 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.625573 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.626203 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.626395 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.626468 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.628690 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:02.628691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:02.628691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:02.628698 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:02.630282 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.630623 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.630624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.630624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.630681 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.630908 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.630909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.630909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.631015 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:02.631211 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:02.631250 (+ 39us) server_negotiation.cc:299] Negotiation successful
0620 09:24:02.631318 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 09:24:02.631407 17886 raft_consensus.cc:374] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:02.631450 17886 raft_consensus.cc:729] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 661d2e6451044ff38adab6b638e43a37, State: Initialized, Role: FOLLOWER
I20260620 09:24:02.631593 17886 consensus_queue.cc:260] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } }
I20260620 09:24:02.631896 17892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } } }
I20260620 09:24:02.631937 17886 sys_catalog.cc:564] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:24:02.631989 17892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:24:02.633330 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260620 09:24:02.633828 17903 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:24:02.633884 17903 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:24:02.635311 17818 master_runner.cc:186] Error getting master registration for 127.4.59.62:40839: OK, instance_id { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" instance_seqno: 1781947442299071 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 09:24:02.638038 17818 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 09:24:02.639915 17736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } attempt: 2
I20260620 09:24:02.643375 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.634649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32936 (local address 127.4.59.62:40839)
0620 09:24:02.634683 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:02.634685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:02.635128 (+ 443us) server_negotiation.cc:407] Connection header received
0620 09:24:02.635220 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:02.635221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:02.635232 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:02.635264 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:02.636073 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.636768 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.637818 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.638105 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.638143 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.640402 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:02.640404 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:02.640404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:02.640412 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:02.642121 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.642576 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.642577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.642578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.642638 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.642821 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.642822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.642822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.642926 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:02.642993 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:02.643198 (+ 205us) server_negotiation.cc:299] Negotiation successful
0620 09:24:02.643284 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":16}
I20260620 09:24:02.643769 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.634689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44692 (local address 127.4.59.60:39605)
0620 09:24:02.634726 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:02.634728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:02.635121 (+ 393us) server_negotiation.cc:407] Connection header received
0620 09:24:02.635229 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:02.635230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:02.635240 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:02.635269 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:02.635994 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.636717 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.637818 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.638105 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.638143 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.639540 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:02.639541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:02.639541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:02.639548 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:02.641240 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.641619 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.641620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.641620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.641672 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.641879 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.641880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.641880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.642036 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:02.642108 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:02.643617 (+ 1509us) server_negotiation.cc:299] Negotiation successful
0620 09:24:02.643686 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:24:02.646395 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.634957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57012 (local address 127.4.59.61:35361)
0620 09:24:02.634992 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:02.634993 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:02.635536 (+ 543us) server_negotiation.cc:407] Connection header received
0620 09:24:02.638472 (+ 2936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:02.638473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:02.638487 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:02.638535 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:02.639210 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.640155 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.641889 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.642168 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.642206 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.643556 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:02.643557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:02.643558 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:02.643564 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:02.645210 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.645559 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.645561 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.645561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.645613 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.645838 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.645839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.645839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.645957 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:02.646032 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:02.646268 (+ 236us) server_negotiation.cc:299] Negotiation successful
0620 09:24:02.646338 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Jun 20 09:24:02 dist-test-slave-414w krb5kdc[17655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for kudu/127.4.59.60@KRBTEST.COM
I20260620 09:24:02.649626 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.640121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39605 (local address 127.0.0.1:44706)
0620 09:24:02.640205 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.640215 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:02.640249 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:02.641199 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:02.641200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:02.641346 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:02.641402 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.641404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.643114 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.643116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.643849 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.643851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.645263 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.645264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.645320 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.645747 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:02.645752 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:02.647142 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:02.648610 (+ 1468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.648611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.648611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.648780 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.648995 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.648995 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.648996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.649050 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.649331 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:02.649332 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:02.649376 (+ 44us) client_negotiation.cc:769] Sending connection context
0620 09:24:02.649444 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 09:24:02.649532 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260620 09:24:02.649925 17912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.640321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44706 (local address 127.4.59.60:39605)
0620 09:24:02.640668 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:02.640671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:02.640678 (+ 7us) server_negotiation.cc:407] Connection header received
0620 09:24:02.640696 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:02.640697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:02.640762 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:02.642205 (+ 1443us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:02.642232 (+ 27us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.643029 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.644865 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.645133 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.645446 (+ 313us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.647253 (+ 1807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:02.647254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:02.647255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:02.647261 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:02.648525 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.648865 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.648866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.648867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.648921 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.649122 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.649122 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.649123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.649261 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:02.649336 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:02.649740 (+ 404us) server_negotiation.cc:299] Negotiation successful
0620 09:24:02.649818 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":80,"threads_started":1}
I20260620 09:24:02.651085 17736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0: Bootstrap starting.
I20260620 09:24:02.651671 17736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:02.651976 17736 log.cc:824] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:02.652688 17736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0: No bootstrap required, opened a new log
I20260620 09:24:02.654137 17736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } }
I20260620 09:24:02.654241 17736 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:02.654263 17736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00994bc94ca947c9af3e68093a8c03d0, State: Initialized, Role: FOLLOWER
I20260620 09:24:02.654423 17736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } }
I20260620 09:24:02.654647 17914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } } }
I20260620 09:24:02.654717 17736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:24:02.654719 17914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 09:24:02.656720 17925 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:24:02.656772 17925 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:24:02.657459 17668 master_runner.cc:186] Error getting master registration for 127.4.59.61:35361: OK, instance_id { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" instance_seqno: 1781947442449504 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 09:24:02.659623 17668 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 09:24:02.683081 17811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } attempt: 3
Jun 20 09:24:02 dist-test-slave-414w krb5kdc[17655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.61@KRBTEST.COM for kudu/127.4.59.60@KRBTEST.COM
I20260620 09:24:02.691497 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.683349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39605 (local address 127.0.0.1:44716)
0620 09:24:02.683411 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:02.683423 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:02.683470 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:02.683791 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:02.683792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:02.683971 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:02.684021 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.684023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.684979 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.684980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.685818 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:02.685821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.686327 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.686328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:02.686387 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.686856 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:02.686862 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:02.688532 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:02.690435 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.690436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.690436 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.690649 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.690859 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:02.690860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:02.690860 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:02.690924 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:02.691185 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:02.691186 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:02.691229 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 09:24:02.691302 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 09:24:02.691393 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260620 09:24:02.691510 17912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:02.683421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44716 (local address 127.4.59.60:39605)
0620 09:24:02.683460 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:02.683462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:02.683470 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:24:02.683519 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:02.683520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:02.683704 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:02.683766 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:02.684109 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.684893 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.685903 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:02.686243 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:02.686328 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:02.688612 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:02.688613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:02.688613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:02.688620 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:02.690348 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.690726 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.690728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.690728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.690787 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:02.690990 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:02.690991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:02.690991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:02.691117 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:02.691183 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:02.691342 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 09:24:02.691420 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 09:24:02.693120 17811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4: Bootstrap starting.
I20260620 09:24:02.693838 17811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:02.694136 17811 log.cc:824] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:02.694970 17811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4: No bootstrap required, opened a new log
I20260620 09:24:02.696902 17811 raft_consensus.cc:348] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } }
I20260620 09:24:02.697034 17811 raft_consensus.cc:374] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:02.697067 17811 raft_consensus.cc:729] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710467195dbd4df8a756e2c0283474f4, State: Initialized, Role: FOLLOWER
I20260620 09:24:02.697218 17811 consensus_queue.cc:260] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } }
I20260620 09:24:02.697507 17928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } } }
I20260620 09:24:02.697573 17811 sys_catalog.cc:564] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:24:02.697654 17928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 09:24:02.699774 17939 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:24:02.699828 17939 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:24:02.701195 17743 master_runner.cc:263] No leader master found from master 710467195dbd4df8a756e2c0283474f4
I20260620 09:24:02.701238 17743 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 09:24:02.797000 17689 tablet.cc:2369] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 09:24:02.844612 17839 tablet.cc:2369] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 09:24:02.947471 17764 tablet.cc:2369] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:24:03.008877 17914 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:24:03.008989 17914 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } }
I20260620 09:24:03.009214 17914 leader_election.cc:288] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710467195dbd4df8a756e2c0283474f4 (127.4.59.61:35361), 661d2e6451044ff38adab6b638e43a37 (127.4.59.60:39605)
I20260620 09:24:03.009493 17860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "00994bc94ca947c9af3e68093a8c03d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661d2e6451044ff38adab6b638e43a37" is_pre_election: true
I20260620 09:24:03.009514 17785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "00994bc94ca947c9af3e68093a8c03d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710467195dbd4df8a756e2c0283474f4" is_pre_election: true
I20260620 09:24:03.009656 17785 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00994bc94ca947c9af3e68093a8c03d0 in term 0.
I20260620 09:24:03.009667 17860 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00994bc94ca947c9af3e68093a8c03d0 in term 0.
I20260620 09:24:03.009845 17686 leader_election.cc:302] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00994bc94ca947c9af3e68093a8c03d0, 710467195dbd4df8a756e2c0283474f4; no voters:
I20260620 09:24:03.010015 17914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:24:03.010068 17914 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:24:03.010102 17914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:03.010826 17914 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } }
I20260620 09:24:03.010965 17914 leader_election.cc:288] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [CANDIDATE]: Term 1 election: Requested vote from peers 710467195dbd4df8a756e2c0283474f4 (127.4.59.61:35361), 661d2e6451044ff38adab6b638e43a37 (127.4.59.60:39605)
I20260620 09:24:03.011107 17785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "00994bc94ca947c9af3e68093a8c03d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710467195dbd4df8a756e2c0283474f4"
I20260620 09:24:03.011147 17860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "00994bc94ca947c9af3e68093a8c03d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661d2e6451044ff38adab6b638e43a37"
I20260620 09:24:03.011221 17785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:03.011222 17860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:03.011935 17860 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00994bc94ca947c9af3e68093a8c03d0 in term 1.
I20260620 09:24:03.012004 17785 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00994bc94ca947c9af3e68093a8c03d0 in term 1.
I20260620 09:24:03.012096 17685 leader_election.cc:302] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00994bc94ca947c9af3e68093a8c03d0, 661d2e6451044ff38adab6b638e43a37; no voters:
I20260620 09:24:03.012203 17914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:24:03.012319 17914 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [term 1 LEADER]: Becoming Leader. State: Replica: 00994bc94ca947c9af3e68093a8c03d0, State: Running, Role: LEADER
I20260620 09:24:03.012420 17914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } }
I20260620 09:24:03.012804 17941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00994bc94ca947c9af3e68093a8c03d0. Latest consensus state: current_term: 1 leader_uuid: "00994bc94ca947c9af3e68093a8c03d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } } }
I20260620 09:24:03.012904 17941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [sys.catalog]: This master's current role is: LEADER
I20260620 09:24:03.013092 17943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:24:03.013314 17943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:24:03.014573 17860 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37 [term 1 FOLLOWER]: Refusing update from remote peer 00994bc94ca947c9af3e68093a8c03d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:24:03.014815 17785 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4 [term 1 FOLLOWER]: Refusing update from remote peer 00994bc94ca947c9af3e68093a8c03d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:24:03.014882 17941 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:24:03.015053 17941 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:24:03.016085 17892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00994bc94ca947c9af3e68093a8c03d0. Latest consensus state: current_term: 1 leader_uuid: "00994bc94ca947c9af3e68093a8c03d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } } }
I20260620 09:24:03.016085 17928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00994bc94ca947c9af3e68093a8c03d0. Latest consensus state: current_term: 1 leader_uuid: "00994bc94ca947c9af3e68093a8c03d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } } }
I20260620 09:24:03.016172 17892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:24:03.016193 17928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:24:03.016921 17914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00994bc94ca947c9af3e68093a8c03d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } } }
I20260620 09:24:03.017006 17914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [sys.catalog]: This master's current role is: LEADER
I20260620 09:24:03.016987 17941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00994bc94ca947c9af3e68093a8c03d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } } }
I20260620 09:24:03.017040 17941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0 [sys.catalog]: This master's current role is: LEADER
I20260620 09:24:03.017220 17928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00994bc94ca947c9af3e68093a8c03d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } } }
I20260620 09:24:03.017293 17928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:24:03.017330 17945 mvcc.cc:205] Tried to move back new op lower bound from 7298856726586122240 to 7298856726579957760. Current Snapshot: MvccSnapshot[applied={T|T < 7298856726586122240}]
I20260620 09:24:03.017387 17943 catalog_manager.cc:1347] Generated new cluster ID: d893070ff4a94ac385ee61ef18647379
I20260620 09:24:03.017427 17943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:24:03.017405 17946 mvcc.cc:205] Tried to move back new op lower bound from 7298856726586122240 to 7298856726579957760. Current Snapshot: MvccSnapshot[applied={T|T < 7298856726586122240}]
I20260620 09:24:03.017477 17892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00994bc94ca947c9af3e68093a8c03d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00994bc94ca947c9af3e68093a8c03d0" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40839 } } peers { permanent_uuid: "710467195dbd4df8a756e2c0283474f4" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 35361 } } peers { permanent_uuid: "661d2e6451044ff38adab6b638e43a37" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39605 } } }
I20260620 09:24:03.017549 17892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:24:03.024506 17943 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:24:03.025115 17943 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:24:03.030418 17943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00994bc94ca947c9af3e68093a8c03d0: Generated new TSK 0
I20260620 09:24:03.030593 17943 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:24:03.634337 17903 catalog_manager.cc:1259] Loaded cluster ID: d893070ff4a94ac385ee61ef18647379
I20260620 09:24:03.634430 17903 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37: loading cluster ID for follower catalog manager: success
I20260620 09:24:03.635301 17903 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37: acquiring CA information for follower catalog manager: success
I20260620 09:24:03.635599 17903 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 661d2e6451044ff38adab6b638e43a37: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 09:24:03.700241 17939 catalog_manager.cc:1259] Loaded cluster ID: d893070ff4a94ac385ee61ef18647379
I20260620 09:24:03.700343 17939 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4: loading cluster ID for follower catalog manager: success
I20260620 09:24:03.701186 17939 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4: acquiring CA information for follower catalog manager: success
I20260620 09:24:03.701494 17939 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 710467195dbd4df8a756e2c0283474f4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 09:24:04.099682 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 17668
I20260620 09:24:04.104722 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 17743
I20260620 09:24:04.109424 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 17818
2026-06-20T09:24:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3087 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:24:04 dist-test-slave-414w krb5kdc[17958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:24:04 dist-test-slave-414w krb5kdc[17958](info): set up 2 sockets
Jun 20 09:24:04 dist-test-slave-414w krb5kdc[17958](info): commencing operation
krb5kdc: starting...
W20260620 09:24:05.145433 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:24:05 dist-test-slave-414w krb5kdc[17958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:24:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:24:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:24:05.262135 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:46661
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:46661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:24:05.341666 17971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:05.341864 17971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:05.341887 17971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:05.342990 17971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:24:05.343025 17971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:05.343042 17971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:05.343055 17971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:24:05.343067 17971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:24:05.344476 17971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:46661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:46661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:05.344664 17971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:05.344831 17971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:05.350005 17977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:05.350111 17976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:05.350219 17979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:05.350301 17971 server_base.cc:1034] running on GCE node
I20260620 09:24:05.350605 17971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:05.350925 17971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:05.352083 17971 hybrid_clock.cc:648] HybridClock initialized: now 1781947445352068 us; error 39 us; skew 500 ppm
Jun 20 09:24:05 dist-test-slave-414w krb5kdc[17958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947445, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:05.354599 17971 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:24:05.355067 17971 webserver.cc:460] Webserver started at http://127.4.59.62:42051/ using document root <none> and password file <none>
I20260620 09:24:05.355293 17971 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:05.355336 17971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:05.355479 17971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:24:05.356544 17971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "9dc618ac701d43beab0cbbd6e7213909"
format_stamp: "Formatted at 2026-06-20 09:24:05 on dist-test-slave-414w"
server_key: "f3c105e1959ebe4426213c00526f6047"
server_key_iv: "7a5dbb0fe9500d0399b2eae8d3b9f5b1"
server_key_version: "encryptionkey@0"
I20260620 09:24:05.356901 17971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "9dc618ac701d43beab0cbbd6e7213909"
format_stamp: "Formatted at 2026-06-20 09:24:05 on dist-test-slave-414w"
server_key: "f3c105e1959ebe4426213c00526f6047"
server_key_iv: "7a5dbb0fe9500d0399b2eae8d3b9f5b1"
server_key_version: "encryptionkey@0"
I20260620 09:24:05.358268 17971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:24:05.359091 17986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:05.359277 17971 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:05.359382 17971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "9dc618ac701d43beab0cbbd6e7213909"
format_stamp: "Formatted at 2026-06-20 09:24:05 on dist-test-slave-414w"
server_key: "f3c105e1959ebe4426213c00526f6047"
server_key_iv: "7a5dbb0fe9500d0399b2eae8d3b9f5b1"
server_key_version: "encryptionkey@0"
I20260620 09:24:05.359470 17971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:05.367408 17971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:05.369561 17971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:05.369719 17971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:05.374509 17971 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:46661
I20260620 09:24:05.374559 18038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:46661 every 8 connection(s)
I20260620 09:24:05.374907 17971 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:24:05.375758 18039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:05.376267 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 17971
I20260620 09:24:05.376379 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:24:05.376561 4332 external_mini_cluster.cc:1428] Setting key d9eb2fcbbfb4946e0c0b162a78454a6d
I20260620 09:24:05.378926 18039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909: Bootstrap starting.
I20260620 09:24:05.379601 18039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:05.379894 18039 log.cc:824] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:05.380633 18039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909: No bootstrap required, opened a new log
I20260620 09:24:05.382943 18039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dc618ac701d43beab0cbbd6e7213909" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46661 } }
I20260620 09:24:05.383121 18039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:05.383158 18039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dc618ac701d43beab0cbbd6e7213909, State: Initialized, Role: FOLLOWER
I20260620 09:24:05.383299 18039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dc618ac701d43beab0cbbd6e7213909" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46661 } }
I20260620 09:24:05.383380 18039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:24:05.383411 18039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:24:05.383457 18039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [term 0 FOLLOWER]: Advancing to term 1
Jun 20 09:24:05 dist-test-slave-414w krb5kdc[17958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:24:05.384198 18039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dc618ac701d43beab0cbbd6e7213909" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46661 } }
I20260620 09:24:05.384338 18039 leader_election.cc:302] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9dc618ac701d43beab0cbbd6e7213909; no voters:
I20260620 09:24:05.384537 18039 leader_election.cc:288] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:24:05.384583 18044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:24:05.384789 18044 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [term 1 LEADER]: Becoming Leader. State: Replica: 9dc618ac701d43beab0cbbd6e7213909, State: Running, Role: LEADER
I20260620 09:24:05.384806 18039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:24:05.384876 18044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dc618ac701d43beab0cbbd6e7213909" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46661 } }
I20260620 09:24:05.385334 18046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9dc618ac701d43beab0cbbd6e7213909" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dc618ac701d43beab0cbbd6e7213909" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46661 } } }
I20260620 09:24:05.385413 18046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [sys.catalog]: This master's current role is: LEADER
I20260620 09:24:05.385218 18045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9dc618ac701d43beab0cbbd6e7213909. Latest consensus state: current_term: 1 leader_uuid: "9dc618ac701d43beab0cbbd6e7213909" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dc618ac701d43beab0cbbd6e7213909" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 46661 } } }
I20260620 09:24:05.385679 18045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909 [sys.catalog]: This master's current role is: LEADER
W20260620 09:24:05.386694 18060 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:24:05.386772 18060 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:24:05.386843 18053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:24:05.387044 18053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:24:05.388415 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:05.377236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54196 (local address 127.4.59.62:46661)
0620 09:24:05.377414 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:05.377424 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:05.377457 (+ 33us) server_negotiation.cc:407] Connection header received
0620 09:24:05.377987 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:05.377995 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:05.378063 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:05.378222 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:05.378798 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:05.380176 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:05.381015 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:05.381638 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:05.381750 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:05.384329 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:05.384334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:05.384334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:05.384353 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:05.386558 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:05.387159 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:05.387162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:05.387162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:05.387266 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:05.387503 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:05.387504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:05.387504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:05.387772 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:05.387912 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:05.388061 (+ 149us) server_negotiation.cc:299] Negotiation successful
0620 09:24:05.388207 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260620 09:24:05.388859 18053 catalog_manager.cc:1347] Generated new cluster ID: bc1e14b35897453d940245d22b616dd6
I20260620 09:24:05.388904 18053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:24:05.389403 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:24:05.400404 18053 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:24:05.400921 18053 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:24:05.406288 18053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9dc618ac701d43beab0cbbd6e7213909: Generated new TSK 0
I20260620 09:24:05.406481 18053 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:24:05.443287 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 17971
2026-06-20T09:24:05Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1332 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:24:05 dist-test-slave-414w krb5kdc[18066](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:24:05 dist-test-slave-414w krb5kdc[18066](info): set up 2 sockets
Jun 20 09:24:05 dist-test-slave-414w krb5kdc[18066](info): commencing operation
krb5kdc: starting...
W20260620 09:24:06.473495 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:24:06 dist-test-slave-414w krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:24:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:24:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:24:06.610368 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:45771
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:45771
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:24:06.692615 18079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:06.692845 18079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:06.692880 18079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:06.694046 18079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:24:06.694111 18079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:06.694139 18079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:06.694169 18079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:24:06.694195 18079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:24:06.696193 18079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:45771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:45771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:06.696476 18079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:06.696686 18079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:06.701386 18084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:06.701431 18085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:06.701419 18087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:06.701550 18079 server_base.cc:1034] running on GCE node
I20260620 09:24:06.701798 18079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:06.702116 18079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:06.703285 18079 hybrid_clock.cc:648] HybridClock initialized: now 1781947446703275 us; error 48 us; skew 500 ppm
Jun 20 09:24:06 dist-test-slave-414w krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947446, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:06.705758 18079 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:24:06.706204 18079 webserver.cc:460] Webserver started at http://127.4.59.62:44053/ using document root <none> and password file <none>
I20260620 09:24:06.706383 18079 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:06.706424 18079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:06.706511 18079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:24:06.707479 18079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "88bad326737143899d009309af50fdac"
format_stamp: "Formatted at 2026-06-20 09:24:06 on dist-test-slave-414w"
server_key: "85feb3b3e2e162603943687daef17258"
server_key_iv: "be39ba34667a790b45393fbf2507d2bf"
server_key_version: "encryptionkey@0"
I20260620 09:24:06.707768 18079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "88bad326737143899d009309af50fdac"
format_stamp: "Formatted at 2026-06-20 09:24:06 on dist-test-slave-414w"
server_key: "85feb3b3e2e162603943687daef17258"
server_key_iv: "be39ba34667a790b45393fbf2507d2bf"
server_key_version: "encryptionkey@0"
I20260620 09:24:06.708963 18079 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:24:06.709721 18094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:06.709905 18079 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:06.709991 18079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "88bad326737143899d009309af50fdac"
format_stamp: "Formatted at 2026-06-20 09:24:06 on dist-test-slave-414w"
server_key: "85feb3b3e2e162603943687daef17258"
server_key_iv: "be39ba34667a790b45393fbf2507d2bf"
server_key_version: "encryptionkey@0"
I20260620 09:24:06.710058 18079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:06.720468 18079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:06.722751 18079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:06.722901 18079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:06.727056 18079 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:45771
I20260620 09:24:06.727161 18146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:45771 every 8 connection(s)
I20260620 09:24:06.727486 18079 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:24:06.728504 18147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:06.730692 18147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac: Bootstrap starting.
I20260620 09:24:06.731216 18147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:06.731446 18147 log.cc:824] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:06.732091 18147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac: No bootstrap required, opened a new log
I20260620 09:24:06.733685 18147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } }
I20260620 09:24:06.733817 18147 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:06.733848 18147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88bad326737143899d009309af50fdac, State: Initialized, Role: FOLLOWER
I20260620 09:24:06.733946 18147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } }
I20260620 09:24:06.734025 18147 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:24:06.734058 18147 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:24:06.734117 18147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:06.734894 18147 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } }
I20260620 09:24:06.735013 18147 leader_election.cc:302] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88bad326737143899d009309af50fdac; no voters:
I20260620 09:24:06.735188 18147 leader_election.cc:288] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:24:06.735289 18150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:24:06.735426 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 18079
I20260620 09:24:06.735441 18150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 1 LEADER]: Becoming Leader. State: Replica: 88bad326737143899d009309af50fdac, State: Running, Role: LEADER
I20260620 09:24:06.735445 18147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:24:06.735569 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:24:06.735561 18150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } }
I20260620 09:24:06.735761 4332 external_mini_cluster.cc:1428] Setting key afd49999c8cb484a1369425784db5872
I20260620 09:24:06.736765 18152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88bad326737143899d009309af50fdac. Latest consensus state: current_term: 1 leader_uuid: "88bad326737143899d009309af50fdac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } } }
I20260620 09:24:06.736843 18151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88bad326737143899d009309af50fdac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } } }
I20260620 09:24:06.736877 18152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [sys.catalog]: This master's current role is: LEADER
I20260620 09:24:06.736899 18151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [sys.catalog]: This master's current role is: LEADER
I20260620 09:24:06.737190 18163 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:24:06.737799 18163 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:24:06.739238 18163 catalog_manager.cc:1347] Generated new cluster ID: bc32a61957304899995a48adb9333cdb
I20260620 09:24:06.739293 18163 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 09:24:06 dist-test-slave-414w krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:24:06.749298 18160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:06.736356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39926 (local address 127.4.59.62:45771)
0620 09:24:06.736804 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:06.736813 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:06.736853 (+ 40us) server_negotiation.cc:407] Connection header received
0620 09:24:06.736980 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:06.736988 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:06.737055 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:06.737239 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:06.738262 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:06.740068 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:06.741317 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:06.742058 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:06.742173 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:06.745111 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:06.745115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:06.745115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:06.745127 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:06.747633 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:06.748051 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:06.748053 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:06.748054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:06.748151 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:06.748364 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:06.748365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:06.748365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:06.748657 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:06.748804 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:06.748925 (+ 121us) server_negotiation.cc:299] Negotiation successful
0620 09:24:06.749050 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":84,"threads_started":1}
I20260620 09:24:06.759090 18163 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:24:06.759596 18163 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:24:06.765697 18163 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac: Generated new TSK 0
I20260620 09:24:06.765892 18163 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:24:06.792097 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45771
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:24:06.871300 18173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:06.871490 18173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:06.871513 18173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:06.872637 18173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:06.872677 18173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:06.872704 18173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:24:06.874122 18173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:06.874372 18173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:06.874603 18173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:06.879454 18179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:06.879444 18178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:06.879441 18181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:06.879725 18173 server_base.cc:1034] running on GCE node
I20260620 09:24:06.879941 18173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:06.880187 18173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:06.881333 18173 hybrid_clock.cc:648] HybridClock initialized: now 1781947446881310 us; error 42 us; skew 500 ppm
Jun 20 09:24:06 dist-test-slave-414w krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947446, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:06.883787 18173 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:24:06.884246 18173 webserver.cc:460] Webserver started at http://127.4.59.1:43423/ using document root <none> and password file <none>
I20260620 09:24:06.884472 18173 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:06.884537 18173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:06.884666 18173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:24:06.885735 18173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "59e50e87043c4072b94658efdebf84aa"
format_stamp: "Formatted at 2026-06-20 09:24:06 on dist-test-slave-414w"
server_key: "167bdda5608daa8c97e29c84c16e0f3f"
server_key_iv: "51d52a45f529d0fe9ddd19fef55a2bbc"
server_key_version: "encryptionkey@0"
I20260620 09:24:06.886093 18173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "59e50e87043c4072b94658efdebf84aa"
format_stamp: "Formatted at 2026-06-20 09:24:06 on dist-test-slave-414w"
server_key: "167bdda5608daa8c97e29c84c16e0f3f"
server_key_iv: "51d52a45f529d0fe9ddd19fef55a2bbc"
server_key_version: "encryptionkey@0"
I20260620 09:24:06.887398 18173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:24:06.888391 18188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:06.888589 18173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:06.888695 18173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "59e50e87043c4072b94658efdebf84aa"
format_stamp: "Formatted at 2026-06-20 09:24:06 on dist-test-slave-414w"
server_key: "167bdda5608daa8c97e29c84c16e0f3f"
server_key_iv: "51d52a45f529d0fe9ddd19fef55a2bbc"
server_key_version: "encryptionkey@0"
I20260620 09:24:06.888782 18173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:06.910574 18173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:06.912467 18173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:06.912626 18173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:06.912869 18173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:24:06.913237 18173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:24:06.913300 18173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:06.913341 18173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:24:06.913368 18173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:06.919242 18173 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:40433
I20260620 09:24:06.919281 18301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:40433 every 8 connection(s)
I20260620 09:24:06.919600 18173 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
Jun 20 09:24:06 dist-test-slave-414w krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947446, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:24:06.926894 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 18173
I20260620 09:24:06.926998 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:24:06.927238 4332 external_mini_cluster.cc:1428] Setting key 3c51f78f4aa780a6bdc8b6aeeb442515
I20260620 09:24:06.928918 18160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:06.920314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:53481 (local address 127.4.59.62:45771)
0620 09:24:06.920438 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:06.920440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:06.920612 (+ 172us) server_negotiation.cc:407] Connection header received
0620 09:24:06.921154 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:06.921156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:06.921167 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:06.921229 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:06.922153 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:06.922675 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:06.923695 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:06.923897 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:06.923982 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:06.926072 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:06.926073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:06.926073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:06.926081 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:06.927575 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:06.927982 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:06.927983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:06.927983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:06.928040 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:06.928283 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:06.928284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:06.928284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:06.928399 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:06.928473 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:06.928741 (+ 268us) server_negotiation.cc:299] Negotiation successful
0620 09:24:06.928822 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 09:24:06.929279 18304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:06.920274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:53481)
0620 09:24:06.920483 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:06.920526 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:06.921031 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:06.921304 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:06.921310 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:06.921870 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:06.922050 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:06.922054 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:06.922777 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:06.922778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:06.923596 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:06.923599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:06.923994 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:06.923995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:06.924114 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:06.924625 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:06.924633 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:06.925975 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:06.927666 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:06.927674 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:06.927676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:06.927894 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:06.928106 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:06.928107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:06.928107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:06.928212 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:06.928465 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:06.928472 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:06.928611 (+ 139us) client_negotiation.cc:769] Sending connection context
0620 09:24:06.928725 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 09:24:06.928882 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":95,"threads_started":1}
I20260620 09:24:06.929708 18302 heartbeater.cc:347] Connected to a master server at 127.4.59.62:45771
I20260620 09:24:06.929805 18302 heartbeater.cc:464] Registering TS with master...
I20260620 09:24:06.929998 18302 heartbeater.cc:511] Master 127.4.59.62:45771 requested a full tablet report, sending...
I20260620 09:24:06.930565 18111 ts_manager.cc:194] Registered new tserver with Master: 59e50e87043c4072b94658efdebf84aa (127.4.59.1:40433)
I20260620 09:24:06.931193 18111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:53481
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:24:06.952073 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45771
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:24:07.031558 18307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:07.031821 18307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:07.031862 18307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:07.033044 18307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:07.033125 18307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:07.033200 18307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:24:07.034703 18307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:07.034963 18307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:07.035197 18307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:07.040392 18315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:07.040534 18312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:07.040416 18313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:07.040632 18307 server_base.cc:1034] running on GCE node
I20260620 09:24:07.040858 18307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:07.041054 18307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:07.042273 18307 hybrid_clock.cc:648] HybridClock initialized: now 1781947447042266 us; error 36 us; skew 500 ppm
Jun 20 09:24:07 dist-test-slave-414w krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:07.044848 18307 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:24:07.045370 18307 webserver.cc:460] Webserver started at http://127.4.59.2:46405/ using document root <none> and password file <none>
I20260620 09:24:07.045611 18307 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:07.045679 18307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:07.045836 18307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:24:07.046953 18307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "425180d7f86e48fc8ddd1611ca8e6ace"
format_stamp: "Formatted at 2026-06-20 09:24:07 on dist-test-slave-414w"
server_key: "03aa33cc0d96897824b860085f7633e8"
server_key_iv: "cce75e5f7627fc9861fdb372a25eb113"
server_key_version: "encryptionkey@0"
I20260620 09:24:07.047324 18307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "425180d7f86e48fc8ddd1611ca8e6ace"
format_stamp: "Formatted at 2026-06-20 09:24:07 on dist-test-slave-414w"
server_key: "03aa33cc0d96897824b860085f7633e8"
server_key_iv: "cce75e5f7627fc9861fdb372a25eb113"
server_key_version: "encryptionkey@0"
I20260620 09:24:07.048671 18307 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:24:07.049551 18322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.049846 18307 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 09:24:07.049952 18307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "425180d7f86e48fc8ddd1611ca8e6ace"
format_stamp: "Formatted at 2026-06-20 09:24:07 on dist-test-slave-414w"
server_key: "03aa33cc0d96897824b860085f7633e8"
server_key_iv: "cce75e5f7627fc9861fdb372a25eb113"
server_key_version: "encryptionkey@0"
I20260620 09:24:07.050028 18307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:07.060065 18307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:07.062196 18307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:07.062359 18307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:07.062592 18307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:24:07.062899 18307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:24:07.062953 18307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.062994 18307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:24:07.063022 18307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.068642 18307 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:40389
I20260620 09:24:07.068740 18435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:40389 every 8 connection(s)
I20260620 09:24:07.069029 18307 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
Jun 20 09:24:07 dist-test-slave-414w krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:24:07.077015 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 18307
I20260620 09:24:07.077174 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:24:07.077400 4332 external_mini_cluster.cc:1428] Setting key 298019e627bca3520e924a22755c19c2
I20260620 09:24:07.078924 18160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.069912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:45617 (local address 127.4.59.62:45771)
0620 09:24:07.070003 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.070005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.070271 (+ 266us) server_negotiation.cc:407] Connection header received
0620 09:24:07.070802 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.070803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.070815 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.070872 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:07.071895 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.072401 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.073496 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.073699 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.073782 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.075918 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:07.075919 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:07.075920 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:07.075927 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:07.077368 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:07.077968 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:07.077969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:07.077969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:07.078017 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:07.078289 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:07.078290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:07.078290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:07.078403 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:07.078478 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.078747 (+ 269us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.078831 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 09:24:07.079128 18438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.069810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:45617)
0620 09:24:07.070127 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:07.070181 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:07.070697 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:07.070954 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:07.070957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:07.071594 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:07.071793 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.071797 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.072522 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.072523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.073403 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.073406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.073793 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.073794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.073914 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.074475 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:07.074483 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:07.075826 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:07.077461 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:07.077468 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:07.077470 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:07.077702 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:07.078106 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:07.078107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:07.078108 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:07.078212 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:07.078470 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:07.078476 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:07.078616 (+ 140us) client_negotiation.cc:769] Sending connection context
0620 09:24:07.078727 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 09:24:07.078885 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":98,"threads_started":1}
I20260620 09:24:07.079550 18436 heartbeater.cc:347] Connected to a master server at 127.4.59.62:45771
I20260620 09:24:07.079656 18436 heartbeater.cc:464] Registering TS with master...
I20260620 09:24:07.079859 18436 heartbeater.cc:511] Master 127.4.59.62:45771 requested a full tablet report, sending...
I20260620 09:24:07.080346 18111 ts_manager.cc:194] Registered new tserver with Master: 425180d7f86e48fc8ddd1611ca8e6ace (127.4.59.2:40389)
I20260620 09:24:07.080804 18111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:45617
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:24:07.103583 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45771
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:24:07.184988 18441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:07.185263 18441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:07.185292 18441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:07.186393 18441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:07.186429 18441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:07.186455 18441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:24:07.188112 18441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:07.188297 18441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:07.188467 18441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:07.193295 18449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:07.193303 18447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:07.193295 18446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:07.193341 18441 server_base.cc:1034] running on GCE node
I20260620 09:24:07.193733 18441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:07.193966 18441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:07.195148 18441 hybrid_clock.cc:648] HybridClock initialized: now 1781947447195125 us; error 34 us; skew 500 ppm
Jun 20 09:24:07 dist-test-slave-414w krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:07.197582 18441 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:24:07.197985 18441 webserver.cc:460] Webserver started at http://127.4.59.3:42535/ using document root <none> and password file <none>
I20260620 09:24:07.198189 18441 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:07.198230 18441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:07.198315 18441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:24:07.199270 18441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "8f0c3394b24b468792befdf0e6948d19"
format_stamp: "Formatted at 2026-06-20 09:24:07 on dist-test-slave-414w"
server_key: "d9f555e1d4a1bca4685835aed3d6a495"
server_key_iv: "6dd5e9389c24b39f0b9180480430b38d"
server_key_version: "encryptionkey@0"
I20260620 09:24:07.199555 18441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "8f0c3394b24b468792befdf0e6948d19"
format_stamp: "Formatted at 2026-06-20 09:24:07 on dist-test-slave-414w"
server_key: "d9f555e1d4a1bca4685835aed3d6a495"
server_key_iv: "6dd5e9389c24b39f0b9180480430b38d"
server_key_version: "encryptionkey@0"
I20260620 09:24:07.200778 18441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:24:07.201557 18456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.201709 18441 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.201794 18441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "8f0c3394b24b468792befdf0e6948d19"
format_stamp: "Formatted at 2026-06-20 09:24:07 on dist-test-slave-414w"
server_key: "d9f555e1d4a1bca4685835aed3d6a495"
server_key_iv: "6dd5e9389c24b39f0b9180480430b38d"
server_key_version: "encryptionkey@0"
I20260620 09:24:07.201846 18441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:07.248450 18441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:07.250828 18441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:07.250988 18441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:07.251237 18441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:24:07.251559 18441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:24:07.251592 18441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.251616 18441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:24:07.251662 18441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.257382 18441 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:44887
I20260620 09:24:07.257432 18569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:44887 every 8 connection(s)
I20260620 09:24:07.257735 18441 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:24:07.258848 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 18441
I20260620 09:24:07.258931 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:24:07.259131 4332 external_mini_cluster.cc:1428] Setting key f3df7fcbfe8b968e42721f84f9fc8ebf
Jun 20 09:24:07 dist-test-slave-414w krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:24:07.267138 18160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.258450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:60879 (local address 127.4.59.62:45771)
0620 09:24:07.258525 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.258526 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.258706 (+ 180us) server_negotiation.cc:407] Connection header received
0620 09:24:07.259292 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.259293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.259306 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.259361 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:07.260395 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.260897 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.261945 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.262152 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.262236 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.264333 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:07.264334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:07.264335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:07.264342 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:07.265621 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:07.266174 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:07.266175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:07.266176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:07.266236 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:07.266491 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:07.266492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:07.266493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:07.266618 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:07.266693 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.266958 (+ 265us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.267044 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 09:24:07.267333 18572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.258406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:60879)
0620 09:24:07.258557 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:07.258598 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:07.259180 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:07.259434 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:07.259438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:07.260107 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:07.260290 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.260294 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.260987 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.260988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.261853 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.261855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.262246 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.262247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.262366 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.262863 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:07.262871 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:07.264234 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:07.265808 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:07.265813 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:07.265815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:07.266017 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:07.266305 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:07.266305 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:07.266306 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:07.266416 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:07.266685 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:07.266692 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:07.266828 (+ 136us) client_negotiation.cc:769] Sending connection context
0620 09:24:07.266936 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 09:24:07.267102 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260620 09:24:07.267752 18570 heartbeater.cc:347] Connected to a master server at 127.4.59.62:45771
I20260620 09:24:07.267838 18570 heartbeater.cc:464] Registering TS with master...
I20260620 09:24:07.268044 18570 heartbeater.cc:511] Master 127.4.59.62:45771 requested a full tablet report, sending...
I20260620 09:24:07.268425 18111 ts_manager.cc:194] Registered new tserver with Master: 8f0c3394b24b468792befdf0e6948d19 (127.4.59.3:44887)
I20260620 09:24:07.268800 18111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:60879
I20260620 09:24:07.270866 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:24:07.286792 18160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.272223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39942 (local address 127.4.59.62:45771)
0620 09:24:07.272290 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.272291 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.272442 (+ 151us) server_negotiation.cc:407] Connection header received
0620 09:24:07.272516 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.272517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.272528 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.272582 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:07.273166 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.273627 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.274369 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.274555 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.274641 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.284310 (+ 9669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:07.284311 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:07.284312 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:07.284317 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:07.285646 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:07.285983 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:07.285984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:07.285984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:07.286040 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:07.286249 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:07.286250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:07.286250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:07.286395 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:07.286502 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.286615 (+ 113us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.286692 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 09:24:07.288259 18111 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:24:07.289911 18111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:24:07.296533 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.292244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:40433 (local address 127.0.0.1:40540)
0620 09:24:07.292633 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:07.292646 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:07.292708 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:07.293016 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:07.293017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:07.293019 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:07.293073 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.293075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.294009 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.294010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.295199 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.295202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.296045 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.296046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.296250 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.296269 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 09:24:07.296386 (+ 117us) client_negotiation.cc:240] Negotiation successful
0620 09:24:07.296448 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":84,"threads_started":1}
I20260620 09:24:07.296730 18582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.292190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:40389 (local address 127.0.0.1:33742)
0620 09:24:07.292352 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:07.292389 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:07.292454 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:07.292900 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:07.292903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:07.292905 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:07.292980 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.292983 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.294413 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.294414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.295580 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.295583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.296462 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.296463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.296536 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.296539 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:24:07.296596 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 09:24:07.296653 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":48,"thread_start_us":57,"threads_started":1}
I20260620 09:24:07.296742 18584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.292309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39462 (local address 127.4.59.3:44887)
0620 09:24:07.292832 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.292836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.292845 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:24:07.292881 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.292883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.292898 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.292967 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:07.293203 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.294315 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.295551 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.296268 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.296344 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.296402 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.296595 (+ 193us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.296647 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":82,"threads_started":1}
I20260620 09:24:07.296883 18586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.292467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40540 (local address 127.4.59.1:40433)
0620 09:24:07.292754 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.292759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.292768 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:24:07.292808 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.292810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.292821 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.292908 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:07.293181 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.293876 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.295306 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.295958 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.296047 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.296110 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.296753 (+ 643us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.296800 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":78,"threads_started":1}
I20260620 09:24:07.297019 18581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.292190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:44887 (local address 127.0.0.1:39462)
0620 09:24:07.292596 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:07.292606 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:07.292654 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:07.293045 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:07.293046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:07.293048 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:07.293084 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.293085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.294409 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.294410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.295461 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.295462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.296357 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.296358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.296430 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.296434 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:24:07.296483 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 09:24:07.296516 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":27,"thread_start_us":76,"threads_started":1}
I20260620 09:24:07.297303 18585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.292437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33742 (local address 127.4.59.2:40389)
0620 09:24:07.292654 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.292659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.292671 (+ 12us) server_negotiation.cc:407] Connection header received
0620 09:24:07.292712 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.292715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.292731 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.293059 (+ 328us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:07.293130 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.294282 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.295679 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.296363 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.296455 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.296506 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.297177 (+ 671us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.297228 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":83,"threads_started":1}
I20260620 09:24:07.297669 18236 tablet_service.cc:1467] Processing CreateTablet for tablet dff90a72b8474799b51542c0170ea40c (DEFAULT_TABLE table=test-table [id=c360b0bda68c4d24a9b1b2e2e06d671c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:24:07.297708 18504 tablet_service.cc:1467] Processing CreateTablet for tablet dff90a72b8474799b51542c0170ea40c (DEFAULT_TABLE table=test-table [id=c360b0bda68c4d24a9b1b2e2e06d671c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:24:07.298038 18236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff90a72b8474799b51542c0170ea40c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:07.298040 18504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff90a72b8474799b51542c0170ea40c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:07.298014 18370 tablet_service.cc:1467] Processing CreateTablet for tablet dff90a72b8474799b51542c0170ea40c (DEFAULT_TABLE table=test-table [id=c360b0bda68c4d24a9b1b2e2e06d671c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:24:07.298288 18370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff90a72b8474799b51542c0170ea40c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:07.300493 18587 tablet_bootstrap.cc:492] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: Bootstrap starting.
I20260620 09:24:07.301105 18587 tablet_bootstrap.cc:654] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:07.301267 18588 tablet_bootstrap.cc:492] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace: Bootstrap starting.
I20260620 09:24:07.301842 18587 log.cc:824] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:07.301930 18588 tablet_bootstrap.cc:654] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:07.302452 18588 log.cc:824] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:07.302608 18587 tablet_bootstrap.cc:492] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: No bootstrap required, opened a new log
I20260620 09:24:07.302680 18587 ts_tablet_manager.cc:1397] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 09:24:07.302789 18589 tablet_bootstrap.cc:492] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19: Bootstrap starting.
I20260620 09:24:07.303184 18588 tablet_bootstrap.cc:492] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace: No bootstrap required, opened a new log
I20260620 09:24:07.303269 18588 ts_tablet_manager.cc:1397] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:24:07.303401 18589 tablet_bootstrap.cc:654] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:07.303921 18589 log.cc:824] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:07.304692 18589 tablet_bootstrap.cc:492] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19: No bootstrap required, opened a new log
I20260620 09:24:07.304769 18589 ts_tablet_manager.cc:1397] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:24:07.304847 18587 raft_consensus.cc:348] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } }
I20260620 09:24:07.305014 18587 raft_consensus.cc:374] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:07.305048 18587 raft_consensus.cc:729] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e50e87043c4072b94658efdebf84aa, State: Initialized, Role: FOLLOWER
I20260620 09:24:07.305231 18587 consensus_queue.cc:260] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } }
I20260620 09:24:07.305338 18588 raft_consensus.cc:348] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } }
I20260620 09:24:07.305505 18587 ts_tablet_manager.cc:1428] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 09:24:07.305506 18588 raft_consensus.cc:374] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:07.305560 18588 raft_consensus.cc:729] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 425180d7f86e48fc8ddd1611ca8e6ace, State: Initialized, Role: FOLLOWER
I20260620 09:24:07.305622 18302 heartbeater.cc:503] Master 127.4.59.62:45771 was elected leader, sending a full tablet report...
I20260620 09:24:07.305675 18588 consensus_queue.cc:260] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } }
I20260620 09:24:07.305944 18588 ts_tablet_manager.cc:1428] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:24:07.306277 18436 heartbeater.cc:503] Master 127.4.59.62:45771 was elected leader, sending a full tablet report...
I20260620 09:24:07.307003 18589 raft_consensus.cc:348] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } }
I20260620 09:24:07.307121 18589 raft_consensus.cc:374] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:07.307143 18589 raft_consensus.cc:729] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f0c3394b24b468792befdf0e6948d19, State: Initialized, Role: FOLLOWER
I20260620 09:24:07.307245 18589 consensus_queue.cc:260] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } }
I20260620 09:24:07.307451 18589 ts_tablet_manager.cc:1428] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:24:07.307508 18570 heartbeater.cc:503] Master 127.4.59.62:45771 was elected leader, sending a full tablet report...
W20260620 09:24:07.319741 18437 tablet.cc:2369] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:24:07.344542 18593 raft_consensus.cc:482] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:24:07.344662 18593 raft_consensus.cc:504] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } }
I20260620 09:24:07.345170 18593 leader_election.cc:288] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 425180d7f86e48fc8ddd1611ca8e6ace (127.4.59.2:40389), 8f0c3394b24b468792befdf0e6948d19 (127.4.59.3:44887)
I20260620 09:24:07.348372 18584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.345456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:38475 (local address 127.4.59.3:44887)
0620 09:24:07.345512 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.345514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.345524 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:24:07.345553 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.345560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.345569 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.345636 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:07.345807 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.346640 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.347590 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.348013 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.348066 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.348075 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.348248 (+ 173us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.348292 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 09:24:07.348379 18304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.345334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:44887 (local address 127.4.59.1:38475)
0620 09:24:07.345398 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:07.345412 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:07.345466 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:07.345643 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:07.345644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:07.345646 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:07.345722 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.345724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.346730 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.346731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.347502 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.347504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.348093 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.348094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.348180 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.348184 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:24:07.348237 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 09:24:07.348282 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 09:24:07.348469 18585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.345438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:39619 (local address 127.4.59.2:40389)
0620 09:24:07.345493 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.345494 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.345717 (+ 223us) server_negotiation.cc:407] Connection header received
0620 09:24:07.345745 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.345746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.345758 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.345805 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:07.345997 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.346749 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.347717 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.348111 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.348180 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.348190 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.348357 (+ 167us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.348397 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 09:24:07.348702 18596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.345334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:40389 (local address 127.4.59.1:39619)
0620 09:24:07.345577 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:07.345585 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:07.345654 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:07.345847 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:07.345847 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:07.345849 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:07.345914 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.345915 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.346826 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.346827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.347591 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.347593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.348189 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.348190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.348274 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.348278 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:24:07.348335 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 09:24:07.348370 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":69,"thread_start_us":97,"threads_started":1}
I20260620 09:24:07.348778 18524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff90a72b8474799b51542c0170ea40c" candidate_uuid: "59e50e87043c4072b94658efdebf84aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f0c3394b24b468792befdf0e6948d19" is_pre_election: true
I20260620 09:24:07.348953 18524 raft_consensus.cc:2454] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59e50e87043c4072b94658efdebf84aa in term 0.
I20260620 09:24:07.349056 18390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff90a72b8474799b51542c0170ea40c" candidate_uuid: "59e50e87043c4072b94658efdebf84aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" is_pre_election: true
I20260620 09:24:07.349208 18190 leader_election.cc:302] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59e50e87043c4072b94658efdebf84aa, 8f0c3394b24b468792befdf0e6948d19; no voters:
I20260620 09:24:07.349236 18390 raft_consensus.cc:2454] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59e50e87043c4072b94658efdebf84aa in term 0.
I20260620 09:24:07.349342 18593 raft_consensus.cc:2781] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:24:07.349398 18593 raft_consensus.cc:482] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:24:07.349421 18593 raft_consensus.cc:3037] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:07.350112 18593 raft_consensus.cc:504] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } }
I20260620 09:24:07.350258 18593 leader_election.cc:288] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [CANDIDATE]: Term 1 election: Requested vote from peers 425180d7f86e48fc8ddd1611ca8e6ace (127.4.59.2:40389), 8f0c3394b24b468792befdf0e6948d19 (127.4.59.3:44887)
I20260620 09:24:07.350404 18390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff90a72b8474799b51542c0170ea40c" candidate_uuid: "59e50e87043c4072b94658efdebf84aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "425180d7f86e48fc8ddd1611ca8e6ace"
I20260620 09:24:07.350426 18524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff90a72b8474799b51542c0170ea40c" candidate_uuid: "59e50e87043c4072b94658efdebf84aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f0c3394b24b468792befdf0e6948d19"
I20260620 09:24:07.350483 18390 raft_consensus.cc:3037] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:07.350497 18524 raft_consensus.cc:3037] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:07.351250 18524 raft_consensus.cc:2454] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59e50e87043c4072b94658efdebf84aa in term 1.
I20260620 09:24:07.351279 18390 raft_consensus.cc:2454] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59e50e87043c4072b94658efdebf84aa in term 1.
I20260620 09:24:07.351434 18190 leader_election.cc:302] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59e50e87043c4072b94658efdebf84aa, 8f0c3394b24b468792befdf0e6948d19; no voters:
I20260620 09:24:07.351554 18593 raft_consensus.cc:2781] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:24:07.351720 18593 raft_consensus.cc:686] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [term 1 LEADER]: Becoming Leader. State: Replica: 59e50e87043c4072b94658efdebf84aa, State: Running, Role: LEADER
I20260620 09:24:07.351826 18593 consensus_queue.cc:237] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } }
I20260620 09:24:07.352663 18110 catalog_manager.cc:5495] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 59e50e87043c4072b94658efdebf84aa (127.4.59.1). New cstate: current_term: 1 leader_uuid: "59e50e87043c4072b94658efdebf84aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } health_report { overall_health: UNKNOWN } } }
I20260620 09:24:07.370026 18586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.367660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40542 (local address 127.4.59.1:40433)
0620 09:24:07.367723 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.367724 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.367734 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:24:07.367784 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.367785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.367803 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.367862 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:24:07.367981 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.368435 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.369218 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.369452 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.369530 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.369671 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:24:07.369775 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:24:07.369844 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.369864 (+ 20us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.369882 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 09:24:07.372242 18390 raft_consensus.cc:1261] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace [term 1 FOLLOWER]: Refusing update from remote peer 59e50e87043c4072b94658efdebf84aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:24:07.372241 18524 raft_consensus.cc:1261] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19 [term 1 FOLLOWER]: Refusing update from remote peer 59e50e87043c4072b94658efdebf84aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:24:07.372627 18597 consensus_queue.cc:1035] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:24:07.372771 18593 consensus_queue.cc:1035] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:24:07.375499 18600 mvcc.cc:205] Tried to move back new op lower bound from 7298856744434520064 to 7298856744354693120. Current Snapshot: MvccSnapshot[applied={T|T < 7298856744434520064}]
I20260620 09:24:07.375720 18602 mvcc.cc:205] Tried to move back new op lower bound from 7298856744434520064 to 7298856744354693120. Current Snapshot: MvccSnapshot[applied={T|T < 7298856744434520064}]
I20260620 09:24:07.375782 18601 mvcc.cc:205] Tried to move back new op lower bound from 7298856744434520064 to 7298856744354693120. Current Snapshot: MvccSnapshot[applied={T|T < 7298856744434520064}]
I20260620 09:24:07.378451 18111 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:24:07.378541 18111 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:24:07.379283 18111 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac: Sending DeleteTablet for 3 replicas of tablet dff90a72b8474799b51542c0170ea40c
I20260620 09:24:07.379568 18236 tablet_service.cc:1514] Processing DeleteTablet for tablet dff90a72b8474799b51542c0170ea40c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:07 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:40540
I20260620 09:24:07.379614 18370 tablet_service.cc:1514] Processing DeleteTablet for tablet dff90a72b8474799b51542c0170ea40c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:07 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:33742
I20260620 09:24:07.379660 18504 tablet_service.cc:1514] Processing DeleteTablet for tablet dff90a72b8474799b51542c0170ea40c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:07 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:39462
I20260620 09:24:07.379832 18611 tablet_replica.cc:330] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace: stopping tablet replica
I20260620 09:24:07.379896 18610 tablet_replica.cc:330] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: stopping tablet replica
I20260620 09:24:07.379935 18611 raft_consensus.cc:2229] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:24:07.380002 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 18173
I20260620 09:24:07.380035 18611 raft_consensus.cc:2258] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:24:07.380231 18610 raft_consensus.cc:2229] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:24:07.380940 18611 ts_tablet_manager.cc:1905] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:24:07.381930 18612 tablet_replica.cc:330] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19: stopping tablet replica
I20260620 09:24:07.382064 18612 raft_consensus.cc:2229] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:24:07.382392 18612 raft_consensus.cc:2258] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:24:07.382789 18612 ts_tablet_manager.cc:1905] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:24:07.382966 18611 ts_tablet_manager.cc:1918] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:24:07.383044 18611 log.cc:1196] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/dff90a72b8474799b51542c0170ea40c
I20260620 09:24:07.383311 18611 ts_tablet_manager.cc:1939] T dff90a72b8474799b51542c0170ea40c P 425180d7f86e48fc8ddd1611ca8e6ace: Deleting consensus metadata
I20260620 09:24:07.383759 18097 catalog_manager.cc:4841] TS 425180d7f86e48fc8ddd1611ca8e6ace (127.4.59.2:40389): tablet dff90a72b8474799b51542c0170ea40c (table test-table [id=c360b0bda68c4d24a9b1b2e2e06d671c]) successfully deleted
I20260620 09:24:07.384213 18612 ts_tablet_manager.cc:1918] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:24:07.384298 18612 log.cc:1196] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/dff90a72b8474799b51542c0170ea40c
I20260620 09:24:07.384541 18612 ts_tablet_manager.cc:1939] T dff90a72b8474799b51542c0170ea40c P 8f0c3394b24b468792befdf0e6948d19: Deleting consensus metadata
I20260620 09:24:07.384917 18096 catalog_manager.cc:4841] TS 8f0c3394b24b468792befdf0e6948d19 (127.4.59.3:44887): tablet dff90a72b8474799b51542c0170ea40c (table test-table [id=c360b0bda68c4d24a9b1b2e2e06d671c]) successfully deleted
W20260620 09:24:07.386744 18098 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.59.1:40433 (error 108)
I20260620 09:24:07.386958 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 18307
I20260620 09:24:07.388980 18581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.388847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:40433 (local address 127.0.0.1:40558)
0620 09:24:07.388911 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:07.388952 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:40433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260620 09:24:07.389240 18098 catalog_manager.cc:4584] TS 59e50e87043c4072b94658efdebf84aa (127.4.59.1:40433): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dff90a72b8474799b51542c0170ea40c: Network error: Client connection negotiation failed: client connection to 127.4.59.1:40433: connect: Connection refused (error 111)
I20260620 09:24:07.391866 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 18441
I20260620 09:24:07.396678 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 18079
I20260620 09:24:07.401825 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:45771
--webserver_interface=127.4.59.62
--webserver_port=44053
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:45771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:24:07.482187 18614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:07.482375 18614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:07.482398 18614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:07.483655 18614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:24:07.483702 18614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:07.483717 18614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:07.483731 18614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:24:07.483743 18614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:24:07.485334 18614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:45771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:45771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=44053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:07.485581 18614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:07.485802 18614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:07.494123 18623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:07.494179 18621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:07.494221 18620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:07.494339 18614 server_base.cc:1034] running on GCE node
I20260620 09:24:07.494591 18614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:07.494894 18614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:07.496038 18614 hybrid_clock.cc:648] HybridClock initialized: now 1781947447496023 us; error 37 us; skew 500 ppm
Jun 20 09:24:07 dist-test-slave-414w krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:07.498487 18614 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:24:07.498850 18614 webserver.cc:460] Webserver started at http://127.4.59.62:44053/ using document root <none> and password file <none>
I20260620 09:24:07.499042 18614 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:07.499095 18614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:07.500420 18614 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:24:07.501055 18630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.501230 18614 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:24:07.501324 18614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "88bad326737143899d009309af50fdac"
format_stamp: "Formatted at 2026-06-20 09:24:06 on dist-test-slave-414w"
server_key: "85feb3b3e2e162603943687daef17258"
server_key_iv: "be39ba34667a790b45393fbf2507d2bf"
server_key_version: "encryptionkey@0"
I20260620 09:24:07.501698 18614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:07.519213 18614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:07.521520 18614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:07.521628 18614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:07.525352 18614 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:45771
I20260620 09:24:07.525421 18682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:45771 every 8 connection(s)
I20260620 09:24:07.525779 18614 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:24:07.526492 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 18614
I20260620 09:24:07.526844 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:40433
--local_ip_for_outbound_sockets=127.4.59.1
--tserver_master_addrs=127.4.59.62:45771
--webserver_port=43423
--webserver_interface=127.4.59.1
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:24:07.528118 18683 sys_catalog.cc:263] Verifying existing consensus state
I20260620 09:24:07.528774 18683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac: Bootstrap starting.
I20260620 09:24:07.536917 18683 log.cc:824] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:07.538892 18683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 09:24:07.539184 18683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac: Bootstrap complete.
I20260620 09:24:07.541545 18683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } }
I20260620 09:24:07.541761 18683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88bad326737143899d009309af50fdac, State: Initialized, Role: FOLLOWER
I20260620 09:24:07.541918 18683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } }
I20260620 09:24:07.541980 18683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:24:07.542047 18683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:24:07.542099 18683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 1 FOLLOWER]: Advancing to term 2
I20260620 09:24:07.543073 18683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } }
I20260620 09:24:07.543161 18683 leader_election.cc:302] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88bad326737143899d009309af50fdac; no voters:
I20260620 09:24:07.543368 18683 leader_election.cc:288] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 09:24:07.543401 18687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 2 FOLLOWER]: Leader election won for term 2
I20260620 09:24:07.543521 18687 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 2 LEADER]: Becoming Leader. State: Replica: 88bad326737143899d009309af50fdac, State: Running, Role: LEADER
I20260620 09:24:07.543627 18683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:24:07.543622 18687 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } }
I20260620 09:24:07.544001 18689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88bad326737143899d009309af50fdac. Latest consensus state: current_term: 2 leader_uuid: "88bad326737143899d009309af50fdac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } } }
I20260620 09:24:07.544082 18689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [sys.catalog]: This master's current role is: LEADER
I20260620 09:24:07.544298 18691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:24:07.544747 18688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "88bad326737143899d009309af50fdac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } } }
I20260620 09:24:07.544879 18688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [sys.catalog]: This master's current role is: LEADER
I20260620 09:24:07.545083 18691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:24:07.545210 18691 catalog_manager.cc:1259] Loaded cluster ID: bc32a61957304899995a48adb9333cdb
I20260620 09:24:07.545482 18691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:24:07.545733 18691 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260620 09:24:07.548517 18691 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260620 09:24:07.550364 18691 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:24:07.550549 18691 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac: Loaded TSK: 0
I20260620 09:24:07.550773 18691 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 09:24:07.620680 18685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:07.620913 18685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:07.620951 18685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:07.622179 18685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:07.622242 18685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:07.622283 18685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:24:07.623732 18685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:40433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=43423
--tserver_master_addrs=127.4.59.62:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:07.623988 18685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:07.624207 18685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:07.629102 18710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:07.629112 18711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:07.629348 18685 server_base.cc:1034] running on GCE node
W20260620 09:24:07.629112 18713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:07.629709 18685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:07.629997 18685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:07.631184 18685 hybrid_clock.cc:648] HybridClock initialized: now 1781947447631170 us; error 29 us; skew 500 ppm
Jun 20 09:24:07 dist-test-slave-414w krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:07.634222 18685 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:24:07.634734 18685 webserver.cc:460] Webserver started at http://127.4.59.1:43423/ using document root <none> and password file <none>
I20260620 09:24:07.634996 18685 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:07.635082 18685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:07.636857 18685 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:24:07.637663 18720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.637859 18685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 09:24:07.637954 18685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "59e50e87043c4072b94658efdebf84aa"
format_stamp: "Formatted at 2026-06-20 09:24:06 on dist-test-slave-414w"
server_key: "167bdda5608daa8c97e29c84c16e0f3f"
server_key_iv: "51d52a45f529d0fe9ddd19fef55a2bbc"
server_key_version: "encryptionkey@0"
I20260620 09:24:07.638394 18685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:07.667311 18685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:07.669890 18685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:07.670061 18685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:07.670344 18685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:24:07.670859 18727 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 09:24:07.672722 18685 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 09:24:07.672784 18685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 09:24:07.672819 18685 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 09:24:07.673686 18685 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 09:24:07.673731 18685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 09:24:07.673894 18727 tablet_bootstrap.cc:492] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: Bootstrap starting.
I20260620 09:24:07.680562 18685 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:40433
I20260620 09:24:07.680721 18834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:40433 every 8 connection(s)
I20260620 09:24:07.680996 18685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:24:07.682410 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 18685
I20260620 09:24:07.682731 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:40389
--local_ip_for_outbound_sockets=127.4.59.2
--tserver_master_addrs=127.4.59.62:45771
--webserver_port=46405
--webserver_interface=127.4.59.2
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:24:07.685026 18727 log.cc:824] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:07.688488 18727 tablet_bootstrap.cc:492] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 09:24:07.688894 18727 tablet_bootstrap.cc:492] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: Bootstrap complete.
Jun 20 09:24:07 dist-test-slave-414w krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:24:07.691445 18727 ts_tablet_manager.cc:1397] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.013s
I20260620 09:24:07.694003 18727 raft_consensus.cc:348] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } }
I20260620 09:24:07.694397 18727 raft_consensus.cc:729] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e50e87043c4072b94658efdebf84aa, State: Initialized, Role: FOLLOWER
I20260620 09:24:07.694548 18727 consensus_queue.cc:260] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } }
I20260620 09:24:07.694867 18727 ts_tablet_manager.cc:1428] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:24:07.696285 18837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.681750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:56835)
0620 09:24:07.682047 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:07.682099 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:07.683637 (+ 1538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:07.684542 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:07.684546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:07.684900 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:07.685191 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.685197 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.686850 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.686851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.688058 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.688061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.688917 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.688918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.689048 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.689697 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:07.689706 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:07.691576 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:07.694254 (+ 2678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:07.694260 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:07.694262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:07.694789 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:07.695135 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:07.695136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:07.695136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:07.695252 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:07.695671 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:07.695677 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:07.695814 (+ 137us) client_negotiation.cc:769] Sending connection context
0620 09:24:07.695908 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 09:24:07.696084 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":87,"threads_started":1}
I20260620 09:24:07.697062 18838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.681835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:56835 (local address 127.4.59.62:45771)
0620 09:24:07.683905 (+ 2070us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.683917 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.684077 (+ 160us) server_negotiation.cc:407] Connection header received
0620 09:24:07.684246 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.684253 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.684322 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.684490 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:07.685331 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.686716 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.688153 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.688722 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.688812 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.691707 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:07.691711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:07.691712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:07.691723 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:07.694146 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:07.694882 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:07.694886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:07.694886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:07.695062 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:07.695316 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:07.695316 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:07.695317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:07.695575 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:07.695645 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.696653 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.696796 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1997,"thread_start_us":79,"threads_started":1}
I20260620 09:24:07.697925 18835 heartbeater.cc:347] Connected to a master server at 127.4.59.62:45771
I20260620 09:24:07.698022 18835 heartbeater.cc:464] Registering TS with master...
I20260620 09:24:07.698179 18835 heartbeater.cc:511] Master 127.4.59.62:45771 requested a full tablet report, sending...
I20260620 09:24:07.698863 18647 ts_manager.cc:194] Registered new tserver with Master: 59e50e87043c4072b94658efdebf84aa (127.4.59.1:40433)
I20260620 09:24:07.700039 18647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:56835
I20260620 09:24:07.701454 18835 heartbeater.cc:503] Master 127.4.59.62:45771 was elected leader, sending a full tablet report...
I20260620 09:24:07.705467 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.700508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:40433 (local address 127.0.0.1:40560)
0620 09:24:07.700937 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:07.700974 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:07.701065 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:07.701474 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:07.701477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:07.701479 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:07.701539 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.701541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.702801 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.702802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.704126 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.704129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.705173 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.705174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.705261 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.705272 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 09:24:07.705340 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 09:24:07.705387 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":93,"threads_started":1}
I20260620 09:24:07.705526 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.700581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40560 (local address 127.4.59.1:40433)
0620 09:24:07.700741 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.700745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.701031 (+ 286us) server_negotiation.cc:407] Connection header received
0620 09:24:07.701180 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.701182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.701340 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.701411 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:07.701615 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.702707 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.704214 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.704976 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.705064 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.705113 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.705379 (+ 266us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.705439 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":75,"threads_started":1}
I20260620 09:24:07.706234 18755 tablet_service.cc:1514] Processing DeleteTablet for tablet dff90a72b8474799b51542c0170ea40c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:07 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:40560
I20260620 09:24:07.706892 18844 tablet_replica.cc:330] stopping tablet replica
I20260620 09:24:07.706997 18844 raft_consensus.cc:2229] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:24:07.707056 18844 raft_consensus.cc:2258] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:24:07.707345 18844 ts_tablet_manager.cc:1905] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:24:07.709586 18844 ts_tablet_manager.cc:1918] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:24:07.709661 18844 log.cc:1196] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal/wals/dff90a72b8474799b51542c0170ea40c
I20260620 09:24:07.709942 18844 ts_tablet_manager.cc:1939] T dff90a72b8474799b51542c0170ea40c P 59e50e87043c4072b94658efdebf84aa: Deleting consensus metadata
I20260620 09:24:07.710415 18634 catalog_manager.cc:4841] TS 59e50e87043c4072b94658efdebf84aa (127.4.59.1:40433): tablet dff90a72b8474799b51542c0170ea40c (table test-table [id=c360b0bda68c4d24a9b1b2e2e06d671c]) successfully deleted
W20260620 09:24:07.773321 18839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:07.773528 18839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:07.773552 18839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:07.774669 18839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:07.774714 18839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:07.774741 18839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:24:07.776696 18839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:40389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=46405
--tserver_master_addrs=127.4.59.62:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:07.776906 18839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:07.777079 18839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:07.781980 18852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:07.782001 18850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:07.781984 18849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:07.782161 18839 server_base.cc:1034] running on GCE node
I20260620 09:24:07.782464 18839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:07.782694 18839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:07.783905 18839 hybrid_clock.cc:648] HybridClock initialized: now 1781947447783867 us; error 66 us; skew 500 ppm
Jun 20 09:24:07 dist-test-slave-414w krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:07.786455 18839 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:24:07.786890 18839 webserver.cc:460] Webserver started at http://127.4.59.2:46405/ using document root <none> and password file <none>
I20260620 09:24:07.787114 18839 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:07.787195 18839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:07.788476 18839 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:07.789052 18859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.789237 18839 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:24:07.789351 18839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "425180d7f86e48fc8ddd1611ca8e6ace"
format_stamp: "Formatted at 2026-06-20 09:24:07 on dist-test-slave-414w"
server_key: "03aa33cc0d96897824b860085f7633e8"
server_key_iv: "cce75e5f7627fc9861fdb372a25eb113"
server_key_version: "encryptionkey@0"
I20260620 09:24:07.789692 18839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:07.806799 18839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:07.808554 18839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:07.808706 18839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:07.808936 18839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:24:07.809309 18839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:24:07.809363 18839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.809406 18839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:24:07.809433 18839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.815172 18839 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:40389
I20260620 09:24:07.815214 18972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:40389 every 8 connection(s)
I20260620 09:24:07.815537 18839 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:24:07.819151 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 18839
I20260620 09:24:07.819459 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:44887
--local_ip_for_outbound_sockets=127.4.59.3
--tserver_master_addrs=127.4.59.62:45771
--webserver_port=42535
--webserver_interface=127.4.59.3
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 09:24:07 dist-test-slave-414w krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:24:07.824997 18838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.816194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:49319 (local address 127.4.59.62:45771)
0620 09:24:07.816234 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.816236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.816450 (+ 214us) server_negotiation.cc:407] Connection header received
0620 09:24:07.816897 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.816898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.816908 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.816963 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:07.817886 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.818357 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.819303 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.819627 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.819698 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.821968 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:07.821969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:07.821969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:07.821975 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:07.823443 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:07.823964 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:07.823965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:07.823966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:07.824024 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:07.824326 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:07.824327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:07.824327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:07.824433 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:07.824493 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.824852 (+ 359us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.824927 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 09:24:07.825296 18975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.816174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:49319)
0620 09:24:07.816326 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:07.816366 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:07.816819 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:07.817012 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:07.817015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:07.817626 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:07.817808 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.817813 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.818444 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.818444 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.819198 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.819200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.819706 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.819707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.819825 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.820327 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:07.820335 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:07.821875 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:07.823524 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:07.823531 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:07.823533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:07.823864 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:07.824106 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:07.824107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:07.824107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:07.824254 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:07.824509 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:07.824515 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:07.824689 (+ 174us) client_negotiation.cc:769] Sending connection context
0620 09:24:07.824828 (+ 139us) client_negotiation.cc:240] Negotiation successful
0620 09:24:07.825003 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260620 09:24:07.825906 18973 heartbeater.cc:347] Connected to a master server at 127.4.59.62:45771
I20260620 09:24:07.826009 18973 heartbeater.cc:464] Registering TS with master...
I20260620 09:24:07.826207 18973 heartbeater.cc:511] Master 127.4.59.62:45771 requested a full tablet report, sending...
I20260620 09:24:07.826574 18647 ts_manager.cc:194] Registered new tserver with Master: 425180d7f86e48fc8ddd1611ca8e6ace (127.4.59.2:40389)
I20260620 09:24:07.826984 18647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:49319
W20260620 09:24:07.902146 18976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:07.902328 18976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:07.902359 18976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:07.903465 18976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:07.903502 18976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:07.903528 18976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:24:07.905086 18976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:44887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=42535
--tserver_master_addrs=127.4.59.62:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:07.905385 18976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:07.905612 18976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:07.910538 18981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:07.910602 18984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:07.910547 18982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:07.911029 18976 server_base.cc:1034] running on GCE node
I20260620 09:24:07.911192 18976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:07.911376 18976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:07.912518 18976 hybrid_clock.cc:648] HybridClock initialized: now 1781947447912508 us; error 30 us; skew 500 ppm
Jun 20 09:24:07 dist-test-slave-414w krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:07.915004 18976 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:24:07.915439 18976 webserver.cc:460] Webserver started at http://127.4.59.3:42535/ using document root <none> and password file <none>
I20260620 09:24:07.915616 18976 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:07.915665 18976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:07.916953 18976 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:24:07.917585 18991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.917752 18976 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.917868 18976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "8f0c3394b24b468792befdf0e6948d19"
format_stamp: "Formatted at 2026-06-20 09:24:07 on dist-test-slave-414w"
server_key: "d9f555e1d4a1bca4685835aed3d6a495"
server_key_iv: "6dd5e9389c24b39f0b9180480430b38d"
server_key_version: "encryptionkey@0"
I20260620 09:24:07.918205 18976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:07.950928 18976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:07.953742 18976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:07.953933 18976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:07.954181 18976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:24:07.954495 18976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:24:07.954551 18976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.954597 18976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:24:07.954620 18976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.960229 18976 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:44887
I20260620 09:24:07.960278 19104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:44887 every 8 connection(s)
I20260620 09:24:07.960595 18976 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:24:07.965050 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 18976
Jun 20 09:24:07 dist-test-slave-414w krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:24:07.970383 18838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.961286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:48285 (local address 127.4.59.62:45771)
0620 09:24:07.961356 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.961358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.961562 (+ 204us) server_negotiation.cc:407] Connection header received
0620 09:24:07.962058 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.962059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.962068 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.962119 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:07.962989 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.963475 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.964398 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.964577 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.964631 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.967192 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:07.967194 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:07.967195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:07.967202 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:07.968711 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:07.969245 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:07.969247 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:07.969247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:07.969310 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:07.969562 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:07.969563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:07.969563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:07.969713 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:07.969813 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.970182 (+ 369us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.970277 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 09:24:07.970577 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.961314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:48285)
0620 09:24:07.961461 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:07.961495 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:07.961963 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:07.962170 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:07.962173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:07.962703 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:07.962905 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.962909 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.963549 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.963550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.964323 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.964324 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.964649 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.964649 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.964742 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.965303 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:07.965311 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:07.967085 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:07.968814 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:07.968820 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:07.968822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:07.969119 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:07.969398 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:07.969399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:07.969399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:07.969496 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:07.969865 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:07.969871 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:07.970029 (+ 158us) client_negotiation.cc:769] Sending connection context
0620 09:24:07.970157 (+ 128us) client_negotiation.cc:240] Negotiation successful
0620 09:24:07.970339 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260620 09:24:07.971135 19105 heartbeater.cc:347] Connected to a master server at 127.4.59.62:45771
I20260620 09:24:07.971283 19105 heartbeater.cc:464] Registering TS with master...
I20260620 09:24:07.971517 19105 heartbeater.cc:511] Master 127.4.59.62:45771 requested a full tablet report, sending...
I20260620 09:24:07.971997 18647 ts_manager.cc:194] Registered new tserver with Master: 8f0c3394b24b468792befdf0e6948d19 (127.4.59.3:44887)
I20260620 09:24:07.972379 18647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:48285
I20260620 09:24:07.973479 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.965474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39956 (local address 127.4.59.62:45771)
0620 09:24:07.965569 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.965570 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.965605 (+ 35us) server_negotiation.cc:407] Connection header received
0620 09:24:07.965680 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.965681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.965689 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.965759 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:07.966407 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.967129 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.968194 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.968394 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.968452 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.970001 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:07.970002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:07.970003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:07.970009 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:07.972007 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:07.972423 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:07.972424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:07.972425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:07.972491 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:07.972901 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:07.972902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:07.972903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:07.973031 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:07.973229 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.973292 (+ 63us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.973373 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":55,"threads_started":1}
I20260620 09:24:07.974227 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:24:07.977018 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.974640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39966 (local address 127.4.59.62:45771)
0620 09:24:07.974711 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.974713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.974744 (+ 31us) server_negotiation.cc:407] Connection header received
0620 09:24:07.974806 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.974807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.974842 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.974914 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:24:07.975002 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.975449 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.976237 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.976419 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.976474 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.976728 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:24:07.976842 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:24:07.976912 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.976954 (+ 42us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.976974 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 09:24:07.977319 18647 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:39966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:24:07.978945 18647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:24:07.980912 18755 tablet_service.cc:1467] Processing CreateTablet for tablet 0ffaaf60c7614c598aae69c1fb245c0a (DEFAULT_TABLE table=test-table [id=22c1c15bdea54e7e97c6be228fd32a31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:24:07.981105 18755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ffaaf60c7614c598aae69c1fb245c0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:07.982533 18727 tablet_bootstrap.cc:492] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa: Bootstrap starting.
I20260620 09:24:07.982959 18727 tablet_bootstrap.cc:654] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:07.983626 18727 tablet_bootstrap.cc:492] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa: No bootstrap required, opened a new log
I20260620 09:24:07.983675 18727 ts_tablet_manager.cc:1397] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:07.983793 18727 raft_consensus.cc:348] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:07.983850 18727 raft_consensus.cc:374] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:07.983892 18727 raft_consensus.cc:729] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e50e87043c4072b94658efdebf84aa, State: Initialized, Role: FOLLOWER
I20260620 09:24:07.983943 18727 consensus_queue.cc:260] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:07.984051 18727 ts_tablet_manager.cc:1428] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:07.985098 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.980902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33754 (local address 127.4.59.2:40389)
0620 09:24:07.981091 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.981095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.981106 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:24:07.981240 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.981249 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.981274 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.981376 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:07.981495 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.982652 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.984030 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.984784 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.984846 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.984886 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.985015 (+ 129us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.985042 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":98,"threads_started":1}
I20260620 09:24:07.985127 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.980628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:40389 (local address 127.0.0.1:33754)
0620 09:24:07.980858 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:07.980868 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:07.980908 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:07.981385 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:07.981386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:07.981388 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:07.981424 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.981426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.982746 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.982746 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.983921 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.983925 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.984870 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.984871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.984931 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.984935 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:24:07.985007 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 09:24:07.985050 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":14,"thread_start_us":81,"threads_started":1}
I20260620 09:24:07.985357 19111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.980605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:44887 (local address 127.0.0.1:39464)
0620 09:24:07.980776 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:07.980787 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:07.980855 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:07.981455 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:07.981456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:07.981458 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:07.981501 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.981502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.982918 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.982920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.984145 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:07.984147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.985119 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.985120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:07.985202 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.985205 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:24:07.985257 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 09:24:07.985291 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260620 09:24:07.985596 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:07.980992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39464 (local address 127.4.59.3:44887)
0620 09:24:07.981278 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:07.981282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:07.981294 (+ 12us) server_negotiation.cc:407] Connection header received
0620 09:24:07.981338 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:07.981341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:07.981355 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:07.981573 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:07.981632 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.982682 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.984328 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:07.984957 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:07.985021 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:07.985074 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:07.985455 (+ 381us) server_negotiation.cc:299] Negotiation successful
0620 09:24:07.985498 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":75,"threads_started":1}
I20260620 09:24:07.986068 18907 tablet_service.cc:1467] Processing CreateTablet for tablet 0ffaaf60c7614c598aae69c1fb245c0a (DEFAULT_TABLE table=test-table [id=22c1c15bdea54e7e97c6be228fd32a31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:24:07.986229 19039 tablet_service.cc:1467] Processing CreateTablet for tablet 0ffaaf60c7614c598aae69c1fb245c0a (DEFAULT_TABLE table=test-table [id=22c1c15bdea54e7e97c6be228fd32a31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:24:07.986449 18907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ffaaf60c7614c598aae69c1fb245c0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:07.986582 19039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ffaaf60c7614c598aae69c1fb245c0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:07.989310 19115 tablet_bootstrap.cc:492] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace: Bootstrap starting.
I20260620 09:24:07.989336 19116 tablet_bootstrap.cc:492] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: Bootstrap starting.
I20260620 09:24:07.989899 19115 tablet_bootstrap.cc:654] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:07.989971 19116 tablet_bootstrap.cc:654] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:07.990375 19115 log.cc:824] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:07.990413 19116 log.cc:824] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:07.991181 19115 tablet_bootstrap.cc:492] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace: No bootstrap required, opened a new log
I20260620 09:24:07.991190 19116 tablet_bootstrap.cc:492] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: No bootstrap required, opened a new log
I20260620 09:24:07.991259 19115 ts_tablet_manager.cc:1397] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:24:07.991271 19116 ts_tablet_manager.cc:1397] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:24:07.993229 19116 raft_consensus.cc:348] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:07.993423 19116 raft_consensus.cc:374] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:07.993330 19115 raft_consensus.cc:348] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:07.993465 19115 raft_consensus.cc:374] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:07.993484 19116 raft_consensus.cc:729] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f0c3394b24b468792befdf0e6948d19, State: Initialized, Role: FOLLOWER
I20260620 09:24:07.993508 19115 raft_consensus.cc:729] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 425180d7f86e48fc8ddd1611ca8e6ace, State: Initialized, Role: FOLLOWER
I20260620 09:24:07.993625 19116 consensus_queue.cc:260] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:07.993625 19115 consensus_queue.cc:260] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:07.993908 19115 ts_tablet_manager.cc:1428] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:24:07.993908 19116 ts_tablet_manager.cc:1428] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:24:07.993947 18973 heartbeater.cc:503] Master 127.4.59.62:45771 was elected leader, sending a full tablet report...
I20260620 09:24:07.994068 19105 heartbeater.cc:503] Master 127.4.59.62:45771 was elected leader, sending a full tablet report...
W20260620 09:24:08.066118 18974 tablet.cc:2369] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 09:24:08.181906 18836 tablet.cc:2369] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 09:24:08.211108 19106 tablet.cc:2369] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:24:08.213068 18841 raft_consensus.cc:482] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:24:08.213224 18841 raft_consensus.cc:504] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:08.213546 18841 leader_election.cc:288] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f0c3394b24b468792befdf0e6948d19 (127.4.59.3:44887), 425180d7f86e48fc8ddd1611ca8e6ace (127.4.59.2:40389)
I20260620 09:24:08.216377 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.213808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:34759 (local address 127.4.59.3:44887)
0620 09:24:08.213850 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:08.213852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:08.213954 (+ 102us) server_negotiation.cc:407] Connection header received
0620 09:24:08.214009 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:08.214010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:08.214023 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:08.214078 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:08.214214 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:08.214680 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:08.215677 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:08.216042 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:08.216096 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:08.216125 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:08.216266 (+ 141us) server_negotiation.cc:299] Negotiation successful
0620 09:24:08.216309 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 09:24:08.216398 19122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.213703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:44887 (local address 127.4.59.1:34759)
0620 09:24:08.213849 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:08.213858 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:08.213945 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:08.214097 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:08.214097 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:08.214099 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:08.214157 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:08.214159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:08.214785 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:08.214786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:08.215580 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:08.215582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:08.216122 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:08.216123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:08.216185 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:08.216189 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:24:08.216241 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 09:24:08.216299 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":31,"thread_start_us":75,"threads_started":1}
I20260620 09:24:08.216416 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.213760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:42113 (local address 127.4.59.2:40389)
0620 09:24:08.213812 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:08.213814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:08.213902 (+ 88us) server_negotiation.cc:407] Connection header received
0620 09:24:08.213943 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:08.213944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:08.213958 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:08.214016 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:08.214173 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:08.214664 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:08.215673 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:08.216052 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:08.216140 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:08.216150 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:08.216316 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 09:24:08.216354 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 09:24:08.216419 19121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.213693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:40389 (local address 127.4.59.1:42113)
0620 09:24:08.213819 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:08.213830 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:08.213893 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:08.214035 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:08.214036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:08.214038 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:08.214098 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:08.214100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:08.214744 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:08.214745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:08.215579 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:08.215580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:08.216153 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:08.216154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:08.216215 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:08.216219 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:24:08.216291 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 09:24:08.216329 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":23,"thread_start_us":53,"threads_started":1}
I20260620 09:24:08.216713 19059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ffaaf60c7614c598aae69c1fb245c0a" candidate_uuid: "59e50e87043c4072b94658efdebf84aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f0c3394b24b468792befdf0e6948d19" is_pre_election: true
I20260620 09:24:08.216830 19059 raft_consensus.cc:2454] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59e50e87043c4072b94658efdebf84aa in term 0.
I20260620 09:24:08.216894 18927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ffaaf60c7614c598aae69c1fb245c0a" candidate_uuid: "59e50e87043c4072b94658efdebf84aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" is_pre_election: true
I20260620 09:24:08.217020 18722 leader_election.cc:302] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59e50e87043c4072b94658efdebf84aa, 8f0c3394b24b468792befdf0e6948d19; no voters:
I20260620 09:24:08.217048 18927 raft_consensus.cc:2454] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59e50e87043c4072b94658efdebf84aa in term 0.
I20260620 09:24:08.217162 18841 raft_consensus.cc:2781] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:24:08.217226 18841 raft_consensus.cc:482] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:24:08.217262 18841 raft_consensus.cc:3037] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:08.217941 18841 raft_consensus.cc:504] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:08.218084 18841 leader_election.cc:288] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [CANDIDATE]: Term 1 election: Requested vote from peers 8f0c3394b24b468792befdf0e6948d19 (127.4.59.3:44887), 425180d7f86e48fc8ddd1611ca8e6ace (127.4.59.2:40389)
I20260620 09:24:08.218250 19059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ffaaf60c7614c598aae69c1fb245c0a" candidate_uuid: "59e50e87043c4072b94658efdebf84aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f0c3394b24b468792befdf0e6948d19"
I20260620 09:24:08.218250 18927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ffaaf60c7614c598aae69c1fb245c0a" candidate_uuid: "59e50e87043c4072b94658efdebf84aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "425180d7f86e48fc8ddd1611ca8e6ace"
I20260620 09:24:08.218327 19059 raft_consensus.cc:3037] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:08.218327 18927 raft_consensus.cc:3037] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:08.219213 19059 raft_consensus.cc:2454] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59e50e87043c4072b94658efdebf84aa in term 1.
I20260620 09:24:08.219213 18927 raft_consensus.cc:2454] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59e50e87043c4072b94658efdebf84aa in term 1.
I20260620 09:24:08.219453 18723 leader_election.cc:302] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 425180d7f86e48fc8ddd1611ca8e6ace, 59e50e87043c4072b94658efdebf84aa; no voters:
I20260620 09:24:08.219558 18841 raft_consensus.cc:2781] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:24:08.219683 18841 raft_consensus.cc:686] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 1 LEADER]: Becoming Leader. State: Replica: 59e50e87043c4072b94658efdebf84aa, State: Running, Role: LEADER
I20260620 09:24:08.219769 18841 consensus_queue.cc:237] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:08.220511 18646 catalog_manager.cc:5495] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 59e50e87043c4072b94658efdebf84aa (127.4.59.1). New cstate: current_term: 1 leader_uuid: "59e50e87043c4072b94658efdebf84aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } health_report { overall_health: HEALTHY } } }
I20260620 09:24:08.266863 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.264236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40572 (local address 127.4.59.1:40433)
0620 09:24:08.264435 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:08.264437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:08.264444 (+ 7us) server_negotiation.cc:407] Connection header received
0620 09:24:08.264481 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:08.264482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:08.264517 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:08.264585 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:24:08.264718 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:08.265270 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:08.265948 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:08.266157 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:08.266209 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:08.266337 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:24:08.266426 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:24:08.266487 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:08.266531 (+ 44us) server_negotiation.cc:299] Negotiation successful
0620 09:24:08.266546 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":132,"threads_started":1}
I20260620 09:24:08.268483 19059 raft_consensus.cc:1261] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19 [term 1 FOLLOWER]: Refusing update from remote peer 59e50e87043c4072b94658efdebf84aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:24:08.268483 18927 raft_consensus.cc:1261] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace [term 1 FOLLOWER]: Refusing update from remote peer 59e50e87043c4072b94658efdebf84aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:24:08.268877 19123 consensus_queue.cc:1035] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:24:08.269032 18841 consensus_queue.cc:1035] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:24:08.271360 19126 mvcc.cc:205] Tried to move back new op lower bound from 7298856748105158656 to 7298856747908952064. Current Snapshot: MvccSnapshot[applied={T|T < 7298856748105158656}]
I20260620 09:24:08.272042 19128 mvcc.cc:205] Tried to move back new op lower bound from 7298856748105158656 to 7298856747908952064. Current Snapshot: MvccSnapshot[applied={T|T < 7298856748105158656}]
I20260620 09:24:08.272143 19127 mvcc.cc:205] Tried to move back new op lower bound from 7298856748105158656 to 7298856747908952064. Current Snapshot: MvccSnapshot[applied={T|T < 7298856748105158656 or (T in {7298856748105158656})}]
I20260620 09:24:08.273406 18647 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:39966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:24:08.273509 18647 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:39966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:24:08.274196 18647 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac: Sending DeleteTablet for 3 replicas of tablet 0ffaaf60c7614c598aae69c1fb245c0a
I20260620 09:24:08.274466 19039 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ffaaf60c7614c598aae69c1fb245c0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:08 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:39464
I20260620 09:24:08.274499 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 18685
I20260620 09:24:08.274515 18907 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ffaaf60c7614c598aae69c1fb245c0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:08 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:33754
I20260620 09:24:08.276010 19136 tablet_replica.cc:330] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: stopping tablet replica
I20260620 09:24:08.276114 19136 raft_consensus.cc:2229] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:24:08.276217 19136 raft_consensus.cc:2258] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:24:08.276638 19136 ts_tablet_manager.cc:1905] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:24:08.278585 19136 ts_tablet_manager.cc:1918] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:24:08.278661 19136 log.cc:1196] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/0ffaaf60c7614c598aae69c1fb245c0a
I20260620 09:24:08.278934 19136 ts_tablet_manager.cc:1939] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: Deleting consensus metadata
I20260620 09:24:08.279312 18632 catalog_manager.cc:4841] TS 8f0c3394b24b468792befdf0e6948d19 (127.4.59.3:44887): tablet 0ffaaf60c7614c598aae69c1fb245c0a (table test-table [id=22c1c15bdea54e7e97c6be228fd32a31]) successfully deleted
W20260620 09:24:08.280563 18634 connection.cc:535] client connection to 127.4.59.1:40433 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 09:24:08.280669 18634 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 09:24:08.280869 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 18839
I20260620 09:24:08.280870 19137 tablet_replica.cc:330] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace: stopping tablet replica
I20260620 09:24:08.285513 19111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.285381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:40433 (local address 127.0.0.1:40586)
0620 09:24:08.285435 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:08.285477 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:40433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 09:24:08.285563 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 18976
I20260620 09:24:08.285625 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.285532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:40389 (local address 127.0.0.1:33766)
0620 09:24:08.285571 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:08.285607 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.2:40389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260620 09:24:08.285662 18634 catalog_manager.cc:4584] TS 59e50e87043c4072b94658efdebf84aa (127.4.59.1:40433): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0ffaaf60c7614c598aae69c1fb245c0a: Network error: Client connection negotiation failed: client connection to 127.4.59.1:40433: connect: Connection refused (error 111)
I20260620 09:24:08.290431 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 18614
I20260620 09:24:08.295639 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:45771
--webserver_interface=127.4.59.62
--webserver_port=44053
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:45771
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:24:08.376888 19139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:08.377096 19139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:08.377120 19139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:08.378291 19139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:24:08.378333 19139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:08.378347 19139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:08.378360 19139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:24:08.378372 19139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:24:08.379765 19139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:45771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:45771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=44053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:08.379949 19139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:08.380120 19139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:08.384773 19147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:08.384848 19144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:08.384941 19139 server_base.cc:1034] running on GCE node
W20260620 09:24:08.384882 19145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:08.385284 19139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:08.385583 19139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:08.386745 19139 hybrid_clock.cc:648] HybridClock initialized: now 1781947448386733 us; error 26 us; skew 500 ppm
Jun 20 09:24:08 dist-test-slave-414w krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947448, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:08.389292 19139 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:24:08.389700 19139 webserver.cc:460] Webserver started at http://127.4.59.62:44053/ using document root <none> and password file <none>
I20260620 09:24:08.389918 19139 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:08.389994 19139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:08.391372 19139 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:08.392143 19154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:08.392345 19139 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:08.392446 19139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "88bad326737143899d009309af50fdac"
format_stamp: "Formatted at 2026-06-20 09:24:06 on dist-test-slave-414w"
server_key: "85feb3b3e2e162603943687daef17258"
server_key_iv: "be39ba34667a790b45393fbf2507d2bf"
server_key_version: "encryptionkey@0"
I20260620 09:24:08.392800 19139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:08.403149 19139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:08.405543 19139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:08.405714 19139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:08.410291 19139 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:45771
I20260620 09:24:08.410351 19206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:45771 every 8 connection(s)
I20260620 09:24:08.410730 19139 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:24:08.412273 19207 sys_catalog.cc:263] Verifying existing consensus state
I20260620 09:24:08.412959 19207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac: Bootstrap starting.
I20260620 09:24:08.420411 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 19139
I20260620 09:24:08.420943 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:40433
--local_ip_for_outbound_sockets=127.4.59.1
--tserver_master_addrs=127.4.59.62:45771
--webserver_port=43423
--webserver_interface=127.4.59.1
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:24:08.421306 19207 log.cc:824] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:08.424649 19207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 09:24:08.425004 19207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac: Bootstrap complete.
I20260620 09:24:08.429095 19207 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } }
I20260620 09:24:08.429540 19207 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88bad326737143899d009309af50fdac, State: Initialized, Role: FOLLOWER
I20260620 09:24:08.429737 19207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } }
I20260620 09:24:08.429826 19207 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:24:08.429888 19207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:24:08.429946 19207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 2 FOLLOWER]: Advancing to term 3
I20260620 09:24:08.431574 19207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } }
I20260620 09:24:08.431702 19207 leader_election.cc:302] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88bad326737143899d009309af50fdac; no voters:
I20260620 09:24:08.431893 19207 leader_election.cc:288] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 09:24:08.432008 19211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 3 FOLLOWER]: Leader election won for term 3
I20260620 09:24:08.432161 19211 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [term 3 LEADER]: Becoming Leader. State: Replica: 88bad326737143899d009309af50fdac, State: Running, Role: LEADER
I20260620 09:24:08.432179 19207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:24:08.432260 19211 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } }
I20260620 09:24:08.432617 19212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "88bad326737143899d009309af50fdac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } } }
I20260620 09:24:08.432690 19212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [sys.catalog]: This master's current role is: LEADER
I20260620 09:24:08.432942 19213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88bad326737143899d009309af50fdac. Latest consensus state: current_term: 3 leader_uuid: "88bad326737143899d009309af50fdac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bad326737143899d009309af50fdac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 45771 } } }
I20260620 09:24:08.433076 19213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88bad326737143899d009309af50fdac [sys.catalog]: This master's current role is: LEADER
I20260620 09:24:08.433126 19218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:24:08.433992 19218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:24:08.434091 19218 catalog_manager.cc:1259] Loaded cluster ID: bc32a61957304899995a48adb9333cdb
I20260620 09:24:08.434134 19218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260620 09:24:08.434378 19218 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6e6d0529ad google::LogMessage::Fail() at ??:0
@ 0x7f6e6d05588a google::LogMessage::SendToLog() at ??:0
@ 0x7f6e6d0524de google::LogMessage::Flush() at ??:0
@ 0x7f6e6d053a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6e74f730f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f6e74f7c8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6e6ef200d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6e6ef15954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6e6fd796db start_thread at ??:0
@ 0x7f6e6bb5061f clone at ??:0
W20260620 09:24:08.520588 19210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:08.520771 19210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:08.520793 19210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:08.521973 19210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:08.522014 19210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:08.522042 19210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:24:08.523612 19210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:40433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=43423
--tserver_master_addrs=127.4.59.62:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:08.523840 19210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:08.524070 19210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:08.528957 19244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:08.529006 19243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:08.529019 19246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:08.529284 19210 server_base.cc:1034] running on GCE node
I20260620 09:24:08.529541 19210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:08.529798 19210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:08.530972 19210 hybrid_clock.cc:648] HybridClock initialized: now 1781947448530953 us; error 34 us; skew 500 ppm
Jun 20 09:24:08 dist-test-slave-414w krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947448, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:08.533964 19210 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:24:08.534365 19210 webserver.cc:460] Webserver started at http://127.4.59.1:43423/ using document root <none> and password file <none>
I20260620 09:24:08.534605 19210 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:08.534651 19210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:08.536160 19210 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 09:24:08.536929 19253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:08.537156 19210 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:24:08.537251 19210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "59e50e87043c4072b94658efdebf84aa"
format_stamp: "Formatted at 2026-06-20 09:24:06 on dist-test-slave-414w"
server_key: "167bdda5608daa8c97e29c84c16e0f3f"
server_key_iv: "51d52a45f529d0fe9ddd19fef55a2bbc"
server_key_version: "encryptionkey@0"
I20260620 09:24:08.537675 19210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:08.556546 19210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:08.559000 19210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:08.559154 19210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:08.559413 19210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:24:08.559973 19260 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 09:24:08.561903 19210 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 09:24:08.561952 19210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 09:24:08.561977 19210 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 09:24:08.562639 19210 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 09:24:08.562696 19210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 09:24:08.562777 19260 tablet_bootstrap.cc:492] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa: Bootstrap starting.
I20260620 09:24:08.569739 19210 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:40433
I20260620 09:24:08.569900 19367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:40433 every 8 connection(s)
I20260620 09:24:08.570164 19210 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:24:08.573684 19260 log.cc:824] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:08.576107 19260 tablet_bootstrap.cc:492] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 09:24:08.576401 19260 tablet_bootstrap.cc:492] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa: Bootstrap complete.
I20260620 09:24:08.577044 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 19210
I20260620 09:24:08.577316 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:40389
--local_ip_for_outbound_sockets=127.4.59.2
--tserver_master_addrs=127.4.59.62:45771
--webserver_port=46405
--webserver_interface=127.4.59.2
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:24:08.577684 19260 ts_tablet_manager.cc:1397] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.006s
I20260620 09:24:08.579202 19260 raft_consensus.cc:348] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:08.579521 19260 raft_consensus.cc:729] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e50e87043c4072b94658efdebf84aa, State: Initialized, Role: FOLLOWER
I20260620 09:24:08.579660 19260 consensus_queue.cc:260] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:08.579914 19260 ts_tablet_manager.cc:1428] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260620 09:24:08.636523 19370 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:24:08.570935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:33085)
0620 09:24:08.571141 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:08.571179 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:08.571725 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:08.636298 (+ 64573us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":115,"threads_started":1}
W20260620 09:24:08.636770 19368 heartbeater.cc:644] Failed to heartbeat to 127.4.59.62:45771 (0 consecutive failures): Network error: Failed to ping master at 127.4.59.62:45771: Client connection negotiation failed: client connection to 127.4.59.62:45771: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 09:24:08.637132 19370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.637021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:49801)
0620 09:24:08.637074 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:08.637115 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 09:24:08.637486 19370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.637388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:46425)
0620 09:24:08.637434 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:08.637460 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 09:24:08.637586 19368 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 09:24:08.660197 19372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:08.660425 19372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:08.660465 19372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:08.661670 19372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:08.661727 19372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:08.661777 19372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:24:08.663250 19372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:40389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=46405
--tserver_master_addrs=127.4.59.62:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:08.663525 19372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:08.663745 19372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:08.668728 19381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:08.668740 19378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:08.668787 19379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:08.668983 19372 server_base.cc:1034] running on GCE node
I20260620 09:24:08.669229 19372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:08.669480 19372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:08.670624 19372 hybrid_clock.cc:648] HybridClock initialized: now 1781947448670598 us; error 50 us; skew 500 ppm
Jun 20 09:24:08 dist-test-slave-414w krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947448, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:08.673096 19372 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:24:08.673583 19372 webserver.cc:460] Webserver started at http://127.4.59.2:46405/ using document root <none> and password file <none>
I20260620 09:24:08.673815 19372 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:08.673859 19372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:08.675271 19372 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:24:08.676205 19388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:08.676484 19372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:24:08.676597 19372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "425180d7f86e48fc8ddd1611ca8e6ace"
format_stamp: "Formatted at 2026-06-20 09:24:07 on dist-test-slave-414w"
server_key: "03aa33cc0d96897824b860085f7633e8"
server_key_iv: "cce75e5f7627fc9861fdb372a25eb113"
server_key_version: "encryptionkey@0"
I20260620 09:24:08.676959 19372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:08.690331 19372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:08.692327 19372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:08.692489 19372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:08.692708 19372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:24:08.693192 19395 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 09:24:08.694628 19372 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 09:24:08.694672 19372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 09:24:08.694700 19372 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 09:24:08.695345 19372 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 09:24:08.695377 19372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 09:24:08.695528 19395 tablet_bootstrap.cc:492] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace: Bootstrap starting.
I20260620 09:24:08.702046 19372 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:40389
I20260620 09:24:08.702131 19502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:40389 every 8 connection(s)
I20260620 09:24:08.702421 19372 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:24:08.702821 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 19372
I20260620 09:24:08.703161 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:44887
--local_ip_for_outbound_sockets=127.4.59.3
--tserver_master_addrs=127.4.59.62:45771
--webserver_port=42535
--webserver_interface=127.4.59.3
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:24:08.705106 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.703193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:40357)
0620 09:24:08.704786 (+ 1593us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:08.704893 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1537,"thread_start_us":95,"threads_started":1}
W20260620 09:24:08.705417 19503 heartbeater.cc:644] Failed to heartbeat to 127.4.59.62:45771 (0 consecutive failures): Network error: Failed to ping master at 127.4.59.62:45771: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
I20260620 09:24:08.705875 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.705742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:56913)
0620 09:24:08.705800 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:08.705838 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 09:24:08.706226 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.706105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:59259)
0620 09:24:08.706161 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:08.706200 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 09:24:08.706408 19503 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 09:24:08.706480 19395 log.cc:824] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:08.708567 19395 tablet_bootstrap.cc:492] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 09:24:08.708859 19395 tablet_bootstrap.cc:492] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace: Bootstrap complete.
I20260620 09:24:08.709846 19395 ts_tablet_manager.cc:1397] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.005s
I20260620 09:24:08.711061 19395 raft_consensus.cc:348] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:08.711333 19395 raft_consensus.cc:729] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 425180d7f86e48fc8ddd1611ca8e6ace, State: Initialized, Role: FOLLOWER
I20260620 09:24:08.711511 19395 consensus_queue.cc:260] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:08.711802 19395 ts_tablet_manager.cc:1428] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260620 09:24:08.712055 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.711938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:50115)
0620 09:24:08.711988 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:08.712025 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 09:24:08.712340 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.712258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:36289)
0620 09:24:08.712287 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:08.712313 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260620 09:24:08.788914 19506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:08.789119 19506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:08.789189 19506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:08.790397 19506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:08.790443 19506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:08.790470 19506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:24:08.791905 19506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:44887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=42535
--tserver_master_addrs=127.4.59.62:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:08.792153 19506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:08.792428 19506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:08.797672 19514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:08.797711 19513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:08.797731 19506 server_base.cc:1034] running on GCE node
W20260620 09:24:08.797675 19516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:08.798084 19506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:08.798286 19506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:08.799489 19506 hybrid_clock.cc:648] HybridClock initialized: now 1781947448799491 us; error 43 us; skew 500 ppm
Jun 20 09:24:08 dist-test-slave-414w krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947448, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:08.802062 19506 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:24:08.802501 19506 webserver.cc:460] Webserver started at http://127.4.59.3:42535/ using document root <none> and password file <none>
I20260620 09:24:08.802681 19506 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:08.802723 19506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:08.803998 19506 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:08.804689 19523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:08.804880 19506 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:08.804968 19506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "8f0c3394b24b468792befdf0e6948d19"
format_stamp: "Formatted at 2026-06-20 09:24:07 on dist-test-slave-414w"
server_key: "d9f555e1d4a1bca4685835aed3d6a495"
server_key_iv: "6dd5e9389c24b39f0b9180480430b38d"
server_key_version: "encryptionkey@0"
I20260620 09:24:08.805347 19506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:08.811975 19506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:08.814185 19506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:08.814302 19506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:08.814491 19506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:24:08.814822 19506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:24:08.814862 19506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:08.814888 19506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:24:08.814939 19506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:08.821230 19506 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:44887
I20260620 09:24:08.821287 19636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:44887 every 8 connection(s)
I20260620 09:24:08.821614 19506 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:24:08.822701 19639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.822303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:53867)
0620 09:24:08.822496 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:08.822567 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":65,"threads_started":1}
W20260620 09:24:08.823117 19637 heartbeater.cc:644] Failed to heartbeat to 127.4.59.62:45771 (0 consecutive failures): Network error: Failed to ping master at 127.4.59.62:45771: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
I20260620 09:24:08.823393 19639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.823287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:56685)
0620 09:24:08.823336 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:08.823369 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 09:24:08.823733 19639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:08.823659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:33809)
0620 09:24:08.823698 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:08.823720 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260620 09:24:08.823851 19637 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 09:24:08.829603 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 19506
I20260620 09:24:09.638236 19641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:09.637942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:39007)
0620 09:24:09.638109 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:09.638173 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":82,"threads_started":1}
I20260620 09:24:09.712939 19642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:09.712704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:41489)
0620 09:24:09.712853 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:09.712889 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":72,"threads_started":1}
I20260620 09:24:09.824386 19643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:09.824162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:57797)
0620 09:24:09.824287 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:09.824334 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":53,"threads_started":1}
I20260620 09:24:10.282218 19644 raft_consensus.cc:482] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:24:10.282330 19644 raft_consensus.cc:504] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:10.282707 19644 leader_election.cc:288] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f0c3394b24b468792befdf0e6948d19 (127.4.59.3:44887), 425180d7f86e48fc8ddd1611ca8e6ace (127.4.59.2:40389)
Jun 20 09:24:10 dist-test-slave-414w krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947448, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.2@KRBTEST.COM
Jun 20 09:24:10 dist-test-slave-414w krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947448, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.3@KRBTEST.COM
I20260620 09:24:10.293009 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:10.282981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:57657 (local address 127.4.59.2:40389)
0620 09:24:10.283181 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:10.283192 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:10.283218 (+ 26us) server_negotiation.cc:407] Connection header received
0620 09:24:10.283312 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:10.283314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:10.283352 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:10.283447 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:10.284234 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:10.285614 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:10.286536 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:10.286930 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:10.287069 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:10.289312 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:10.289316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:10.289316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:10.289333 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:10.291228 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:10.291813 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:10.291816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:10.291816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:10.291911 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:10.292164 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:10.292165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:10.292165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:10.292393 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:10.292498 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:10.292782 (+ 284us) server_negotiation.cc:299] Negotiation successful
0620 09:24:10.292904 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":66,"threads_started":1}
I20260620 09:24:10.293009 19645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:10.282857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:40389 (local address 127.4.59.1:57657)
0620 09:24:10.282994 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:10.283006 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:10.283066 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:10.283512 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:10.283515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:10.283895 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:10.284131 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:10.284137 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:10.285707 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:10.285708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:10.286439 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:10.286441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:10.287031 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:10.287032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:10.287133 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:10.287753 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:10.287761 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:10.289187 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:10.291313 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:10.291319 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:10.291320 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:10.291676 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:10.291989 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:10.291990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:10.291990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:10.292091 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:10.292478 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:10.292484 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:10.292617 (+ 133us) client_negotiation.cc:769] Sending connection context
0620 09:24:10.292718 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 09:24:10.292886 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":51,"thread_start_us":45,"threads_started":1}
I20260620 09:24:10.293119 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:10.283016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:44007 (local address 127.4.59.3:44887)
0620 09:24:10.283169 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:10.283180 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:10.283209 (+ 29us) server_negotiation.cc:407] Connection header received
0620 09:24:10.283782 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:10.283785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:10.283831 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:10.283961 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:10.284271 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:10.285593 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:10.286548 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:10.286931 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:10.287050 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:10.289705 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:10.289708 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:10.289709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:10.289717 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:10.291469 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:10.291861 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:10.291864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:10.291865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:10.291958 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:10.292187 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:10.292188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:10.292188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:10.292404 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:10.292501 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:10.292884 (+ 383us) server_negotiation.cc:299] Negotiation successful
0620 09:24:10.293009 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":54,"threads_started":1}
I20260620 09:24:10.293202 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:10.282903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:44887 (local address 127.4.59.1:44007)
0620 09:24:10.283007 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:10.283017 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:10.283074 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:10.283956 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:10.283956 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:10.284117 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:10.284155 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:10.284157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:10.285668 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:10.285669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:10.286438 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:10.286440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:10.287022 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:10.287023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:10.287133 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:10.287776 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:10.287780 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:10.289574 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:10.291546 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:10.291546 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:10.291547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:10.291796 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:10.292028 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:10.292029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:10.292029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:10.292112 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:10.292498 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:10.292499 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:10.292617 (+ 118us) client_negotiation.cc:769] Sending connection context
0620 09:24:10.292817 (+ 200us) client_negotiation.cc:240] Negotiation successful
0620 09:24:10.292959 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":30,"thread_start_us":50,"threads_started":1}
I20260620 09:24:10.294008 19457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ffaaf60c7614c598aae69c1fb245c0a" candidate_uuid: "59e50e87043c4072b94658efdebf84aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" is_pre_election: true
I20260620 09:24:10.294171 19457 raft_consensus.cc:2454] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59e50e87043c4072b94658efdebf84aa in term 1.
W20260620 09:24:10.294220 19255 leader_election.cc:341] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f0c3394b24b468792befdf0e6948d19 (127.4.59.3:44887): Not found: Tablet not found: 0ffaaf60c7614c598aae69c1fb245c0a
I20260620 09:24:10.294425 19256 leader_election.cc:302] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 425180d7f86e48fc8ddd1611ca8e6ace, 59e50e87043c4072b94658efdebf84aa; no voters: 8f0c3394b24b468792befdf0e6948d19
I20260620 09:24:10.294541 19644 raft_consensus.cc:2781] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 09:24:10.294600 19644 raft_consensus.cc:482] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:24:10.294627 19644 raft_consensus.cc:3037] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 1 FOLLOWER]: Advancing to term 2
I20260620 09:24:10.295753 19644 raft_consensus.cc:504] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:10.295917 19644 leader_election.cc:288] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [CANDIDATE]: Term 2 election: Requested vote from peers 8f0c3394b24b468792befdf0e6948d19 (127.4.59.3:44887), 425180d7f86e48fc8ddd1611ca8e6ace (127.4.59.2:40389)
I20260620 09:24:10.296104 19457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ffaaf60c7614c598aae69c1fb245c0a" candidate_uuid: "59e50e87043c4072b94658efdebf84aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "425180d7f86e48fc8ddd1611ca8e6ace"
I20260620 09:24:10.296188 19457 raft_consensus.cc:3037] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace [term 1 FOLLOWER]: Advancing to term 2
I20260620 09:24:10.297230 19457 raft_consensus.cc:2454] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59e50e87043c4072b94658efdebf84aa in term 2.
W20260620 09:24:10.297281 19255 leader_election.cc:341] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8f0c3394b24b468792befdf0e6948d19 (127.4.59.3:44887): Not found: Tablet not found: 0ffaaf60c7614c598aae69c1fb245c0a
I20260620 09:24:10.297389 19256 leader_election.cc:302] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 425180d7f86e48fc8ddd1611ca8e6ace, 59e50e87043c4072b94658efdebf84aa; no voters: 8f0c3394b24b468792befdf0e6948d19
I20260620 09:24:10.297508 19644 raft_consensus.cc:2781] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 2 FOLLOWER]: Leader election won for term 2
I20260620 09:24:10.297742 19644 raft_consensus.cc:686] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [term 2 LEADER]: Becoming Leader. State: Replica: 59e50e87043c4072b94658efdebf84aa, State: Running, Role: LEADER
I20260620 09:24:10.297878 19644 consensus_queue.cc:237] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:10.298038 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:10.297924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:42449)
0620 09:24:10.297978 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:10.298012 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 09:24:10.713574 19651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:10.713388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:47935)
0620 09:24:10.713494 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:10.713525 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
W20260620 09:24:10.762204 19255 consensus_peers.cc:487] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa -> Peer 8f0c3394b24b468792befdf0e6948d19 (127.4.59.3:44887): Couldn't send request to peer 8f0c3394b24b468792befdf0e6948d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0ffaaf60c7614c598aae69c1fb245c0a. This is attempt 1: this message will repeat every 5th retry.
I20260620 09:24:10.821074 19457 raft_consensus.cc:1261] T 0ffaaf60c7614c598aae69c1fb245c0a P 425180d7f86e48fc8ddd1611ca8e6ace [term 2 FOLLOWER]: Refusing update from remote peer 59e50e87043c4072b94658efdebf84aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260620 09:24:10.821564 19644 consensus_queue.cc:1035] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260620 09:24:10.822548 19651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:10.822402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:44721)
0620 09:24:10.822479 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:10.822517 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 09:24:10.823493 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:10.823294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:37653)
0620 09:24:10.823402 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:10.823437 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":42,"threads_started":1}
I20260620 09:24:10.823604 19651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:10.823463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:60455)
0620 09:24:10.823522 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:10.823571 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 09:24:10.825063 19658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:10.824870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:34579)
0620 09:24:10.824991 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:10.825024 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":83,"threads_started":1}
I20260620 09:24:11.150719 19659 ts_tablet_manager.cc:927] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: Initiating tablet copy from peer 59e50e87043c4072b94658efdebf84aa (127.4.59.1:40433)
I20260620 09:24:11.150974 19659 tablet_copy_client.cc:316] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: tablet copy: Beginning tablet copy session from remote peer at address 127.4.59.1:40433
Jun 20 09:24:11 dist-test-slave-414w krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947448, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.1@KRBTEST.COM
I20260620 09:24:11.158674 19661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:11.151402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:59407 (local address 127.4.59.1:40433)
0620 09:24:11.151587 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:11.151592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:11.151601 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:24:11.151624 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:11.151625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:11.151638 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:11.151702 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:11.152160 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:11.152866 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:11.153738 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:11.154027 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:11.154115 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:11.156053 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:11.156060 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:11.156060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:11.156067 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:11.157676 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:11.158012 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:11.158014 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:11.158015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:11.158068 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:11.158231 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:11.158232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:11.158232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:11.158362 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:11.158423 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:11.158553 (+ 130us) server_negotiation.cc:299] Negotiation successful
0620 09:24:11.158607 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":101,"threads_started":1}
I20260620 09:24:11.158685 19658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:11.151307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:40433 (local address 127.4.59.3:59407)
0620 09:24:11.151383 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:11.151402 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:11.151478 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:11.151729 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:11.151734 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:11.152004 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:11.152084 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:11.152085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:11.152940 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:11.152941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:11.153585 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:11.153587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:11.154128 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:11.154128 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:11.154178 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:11.154638 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:11.154643 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:11.155979 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:11.157750 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:11.157756 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:11.157757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:11.157939 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:11.158128 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:11.158128 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:11.158129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:11.158179 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:11.158421 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:11.158427 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:11.158469 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 09:24:11.158532 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 09:24:11.158613 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260620 09:24:11.159157 19342 tablet_copy_service.cc:140] P 59e50e87043c4072b94658efdebf84aa: Received BeginTabletCopySession request for tablet 0ffaaf60c7614c598aae69c1fb245c0a from peer 8f0c3394b24b468792befdf0e6948d19 ({username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:59407)
I20260620 09:24:11.159257 19342 tablet_copy_service.cc:161] P 59e50e87043c4072b94658efdebf84aa: Beginning new tablet copy session on tablet 0ffaaf60c7614c598aae69c1fb245c0a from peer 8f0c3394b24b468792befdf0e6948d19 at {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:59407: session id = 8f0c3394b24b468792befdf0e6948d19-0ffaaf60c7614c598aae69c1fb245c0a
I20260620 09:24:11.159756 19342 tablet_copy_source_session.cc:205] T 0ffaaf60c7614c598aae69c1fb245c0a P 59e50e87043c4072b94658efdebf84aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 09:24:11.160717 19659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ffaaf60c7614c598aae69c1fb245c0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:11.163291 19659 tablet_copy_client.cc:791] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: tablet copy: Starting download of 0 data blocks...
I20260620 09:24:11.163431 19659 tablet_copy_client.cc:655] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: tablet copy: Starting download of 1 WAL segments...
I20260620 09:24:11.164178 19659 tablet_copy_client.cc:531] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 09:24:11.164772 19659 tablet_bootstrap.cc:492] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: Bootstrap starting.
I20260620 09:24:11.166042 19659 log.cc:824] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:11.168294 19659 tablet_bootstrap.cc:492] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 09:24:11.168468 19659 tablet_bootstrap.cc:492] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: Bootstrap complete.
I20260620 09:24:11.168565 19659 ts_tablet_manager.cc:1397] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 09:24:11.169469 19659 raft_consensus.cc:348] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:11.169689 19659 raft_consensus.cc:729] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f0c3394b24b468792befdf0e6948d19, State: Initialized, Role: FOLLOWER
I20260620 09:24:11.169821 19659 consensus_queue.cc:260] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f0c3394b24b468792befdf0e6948d19" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 44887 } } peers { permanent_uuid: "425180d7f86e48fc8ddd1611ca8e6ace" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 40389 } } peers { permanent_uuid: "59e50e87043c4072b94658efdebf84aa" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 40433 } }
I20260620 09:24:11.170094 19659 ts_tablet_manager.cc:1428] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20260620 09:24:11.170347 19658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:11.170243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:45981)
0620 09:24:11.170292 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:11.170321 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 09:24:11.170382 19342 tablet_copy_service.cc:342] P 59e50e87043c4072b94658efdebf84aa: Request end of tablet copy session 8f0c3394b24b468792befdf0e6948d19-0ffaaf60c7614c598aae69c1fb245c0a received from {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:59407
I20260620 09:24:11.170542 19342 tablet_copy_service.cc:433] P 59e50e87043c4072b94658efdebf84aa: ending tablet copy session 8f0c3394b24b468792befdf0e6948d19-0ffaaf60c7614c598aae69c1fb245c0a on tablet 0ffaaf60c7614c598aae69c1fb245c0a with peer 8f0c3394b24b468792befdf0e6948d19
I20260620 09:24:11.170715 19658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:11.170629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:51279)
0620 09:24:11.170665 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:11.170687 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260620 09:24:11.536906 19666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:11.536756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:43087)
0620 09:24:11.536838 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:11.536861 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70,"thread_start_us":43,"threads_started":1}
I20260620 09:24:11.536942 19591 raft_consensus.cc:1203] T 0ffaaf60c7614c598aae69c1fb245c0a P 8f0c3394b24b468792befdf0e6948d19 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260620 09:24:11.537518 19658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:11.537357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:54177)
0620 09:24:11.537445 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:11.537482 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260620 09:24:11.824179 19667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:11.823991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:38059)
0620 09:24:11.824107 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:11.824141 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
I20260620 09:24:12.537588 19668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:12.537370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:35635)
0620 09:24:12.537492 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:12.537529 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260620 09:24:12.538121 19669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:12.537942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:57351)
0620 09:24:12.538051 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:12.538085 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":46,"threads_started":1}
I20260620 09:24:12.824801 19670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:12.824619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:35105)
0620 09:24:12.824731 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:12.824764 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":53,"threads_started":1}
I20260620 09:24:13.538221 19672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:13.538025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:44471)
0620 09:24:13.538139 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:13.538173 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260620 09:24:13.538704 19673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:13.538525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:48945)
0620 09:24:13.538632 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:13.538667 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":64,"threads_started":1}
I20260620 09:24:13.825588 19674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:13.825364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:39069)
0620 09:24:13.825512 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:13.825548 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
I20260620 09:24:14.538966 19675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:14.538742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:37641)
0620 09:24:14.538879 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:14.538919 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260620 09:24:14.539378 19675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:14.539279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:33509)
0620 09:24:14.539326 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:14.539357 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 09:24:14.539436 19676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:14.539234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:48297)
0620 09:24:14.539361 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:14.539397 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260620 09:24:14.826300 19677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:14.826102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:45345)
0620 09:24:14.826230 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:14.826261 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260620 09:24:15.540050 19678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:15.539811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:42761)
0620 09:24:15.539960 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:15.539995 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260620 09:24:15.540207 19679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:15.539977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:55545)
0620 09:24:15.540102 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:15.540152 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260620 09:24:15.826989 19680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:15.826764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:38171)
0620 09:24:15.826914 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:15.826951 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":58,"threads_started":1}
I20260620 09:24:16.540691 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:16.540469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:35595)
0620 09:24:16.540591 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:16.540628 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
I20260620 09:24:16.540846 19682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:16.540599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:40197)
0620 09:24:16.540760 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:16.540799 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":72,"threads_started":1}
I20260620 09:24:16.827823 19683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:16.827584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:34729)
0620 09:24:16.827729 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:16.827775 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":73,"threads_started":1}
I20260620 09:24:17.541508 19684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:17.541261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:54735)
0620 09:24:17.541423 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:17.541461 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
I20260620 09:24:17.541550 19685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:17.541281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:51625)
0620 09:24:17.541418 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:17.541498 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260620 09:24:17.828545 19686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:17.828339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:49633)
0620 09:24:17.828464 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:17.828501 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
I20260620 09:24:18.542209 19687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:18.541990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:39849)
0620 09:24:18.542118 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:18.542160 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260620 09:24:18.542232 19688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:18.542030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:48565)
0620 09:24:18.542154 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:18.542191 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260620 09:24:18.829308 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:18.829030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:47985)
0620 09:24:18.829220 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:18.829263 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":74,"threads_started":1}
I20260620 09:24:19.543071 19691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:19.542837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:47735)
0620 09:24:19.542989 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:19.543024 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":76,"threads_started":1}
I20260620 09:24:19.543071 19690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:19.542837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:49309)
0620 09:24:19.542987 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:19.543026 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260620 09:24:19.830035 19692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:19.829779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:50367)
0620 09:24:19.829953 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:19.829988 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":94,"threads_started":1}
I20260620 09:24:20.543720 19693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:20.543478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:47393)
0620 09:24:20.543613 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:20.543652 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260620 09:24:20.543871 19694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:20.543636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:47133)
0620 09:24:20.543770 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:20.543804 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
I20260620 09:24:20.830727 19695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:20.830525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:59093)
0620 09:24:20.830643 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:20.830686 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
I20260620 09:24:21.544409 19696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:21.544195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:56087)
0620 09:24:21.544332 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:21.544363 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260620 09:24:21.544549 19697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:21.544318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:47391)
0620 09:24:21.544445 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:21.544483 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260620 09:24:21.831374 19698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:21.831166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:36959)
0620 09:24:21.831295 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:21.831333 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":56,"threads_started":1}
I20260620 09:24:22.545238 19699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:22.544971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:50449)
0620 09:24:22.545106 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:22.545174 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
I20260620 09:24:22.545312 19700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:22.545048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:54349)
0620 09:24:22.545244 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:22.545273 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":70,"threads_started":1}
I20260620 09:24:22.832028 19701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:22.831823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:33451)
0620 09:24:22.831943 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:22.831984 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260620 09:24:23.545989 19702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:23.545754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.3:44117)
0620 09:24:23.545902 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:23.545938 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
I20260620 09:24:23.546007 19703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:23.545774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.1:48883)
0620 09:24:23.545911 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:23.545944 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260620 09:24:23.832798 19704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:23.832575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:45771 (local address 127.4.59.2:59719)
0620 09:24:23.832715 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:23.832752 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:45771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":60,"threads_started":1}
I20260620 09:24:23.838055 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 19210
I20260620 09:24:23.842861 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 19372
I20260620 09:24:23.847517 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 19506
2026-06-20T09:24:23Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18407 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:24:23 dist-test-slave-414w krb5kdc[19707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:24:23 dist-test-slave-414w krb5kdc[19707](info): set up 2 sockets
Jun 20 09:24:23 dist-test-slave-414w krb5kdc[19707](info): commencing operation
krb5kdc: starting...
W20260620 09:24:24.889441 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:24:24 dist-test-slave-414w krb5kdc[19707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:24:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:24:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:24:25.007647 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:35489
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:35489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:24:25.088434 19720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:25.088657 19720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:25.088696 19720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:25.089869 19720 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 09:24:25.089932 19720 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 09:24:25.089956 19720 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 09:24:25.089986 19720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:24:25.090021 19720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:25.090052 19720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:25.090075 19720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:24:25.090101 19720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:24:25.091544 19720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:35489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:35489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:25.091814 19720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:25.092033 19720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 09:24:25.097018 19720 server_base.cc:1034] running on GCE node
W20260620 09:24:25.097131 19726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:25.097131 19728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:25.097324 19725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:25.097572 19720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:25.097909 19720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:25.099095 19720 hybrid_clock.cc:648] HybridClock initialized: now 1781947465099085 us; error 36 us; skew 500 ppm
Jun 20 09:24:25 dist-test-slave-414w krb5kdc[19707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947465, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:25.101437 19720 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:24:25.101866 19720 webserver.cc:460] Webserver started at http://127.4.59.62:43619/ using document root <none> and password file <none>
I20260620 09:24:25.102095 19720 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:25.102160 19720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:25.102312 19720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:24:25.103291 19720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "44493f35a1514b2ba46b0e3d05665518"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "faae8e09901bf012f1d3610bc33f4470"
server_key_iv: "625ffa47380b5158e9470664a05d3a85"
server_key_version: "encryptionkey@0"
I20260620 09:24:25.103638 19720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "44493f35a1514b2ba46b0e3d05665518"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "faae8e09901bf012f1d3610bc33f4470"
server_key_iv: "625ffa47380b5158e9470664a05d3a85"
server_key_version: "encryptionkey@0"
I20260620 09:24:25.104898 19720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:24:25.105695 19735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:25.105877 19720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:25.105970 19720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "44493f35a1514b2ba46b0e3d05665518"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "faae8e09901bf012f1d3610bc33f4470"
server_key_iv: "625ffa47380b5158e9470664a05d3a85"
server_key_version: "encryptionkey@0"
I20260620 09:24:25.106055 19720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:25.120113 19720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:25.122208 19720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:25.122362 19720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:25.126355 19720 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:35489
I20260620 09:24:25.126391 19787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:35489 every 8 connection(s)
I20260620 09:24:25.126730 19720 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:24:25.127532 19788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:25.129751 19788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Bootstrap starting.
I20260620 09:24:25.130311 19788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:25.130600 19788 log.cc:824] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:25.131240 19788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: No bootstrap required, opened a new log
I20260620 09:24:25.132017 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 19720
I20260620 09:24:25.132129 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:24:25.132313 4332 external_mini_cluster.cc:1428] Setting key d084a423ba31da38dbf94b21e9156e5a
I20260620 09:24:25.133212 19788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } }
I20260620 09:24:25.133374 19788 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:25.133414 19788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44493f35a1514b2ba46b0e3d05665518, State: Initialized, Role: FOLLOWER
I20260620 09:24:25.133580 19788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } }
I20260620 09:24:25.133666 19788 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:24:25.133689 19788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:24:25.133754 19788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:25.134781 19788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } }
I20260620 09:24:25.134886 19788 leader_election.cc:302] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44493f35a1514b2ba46b0e3d05665518; no voters:
I20260620 09:24:25.135030 19788 leader_election.cc:288] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:24:25.135104 19793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:24:25.135236 19793 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 1 LEADER]: Becoming Leader. State: Replica: 44493f35a1514b2ba46b0e3d05665518, State: Running, Role: LEADER
I20260620 09:24:25.135246 19788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:24:25.135375 19793 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } }
I20260620 09:24:25.135747 19793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44493f35a1514b2ba46b0e3d05665518. Latest consensus state: current_term: 1 leader_uuid: "44493f35a1514b2ba46b0e3d05665518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } } }
I20260620 09:24:25.135881 19793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [sys.catalog]: This master's current role is: LEADER
I20260620 09:24:25.136148 19801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:24:25.136684 19801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:24:25.138322 19801 catalog_manager.cc:1347] Generated new cluster ID: a588143cad7c4a15bdbabdb94789f318
I20260620 09:24:25.139129 19795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44493f35a1514b2ba46b0e3d05665518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } } }
I20260620 09:24:25.139183 19801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:24:25.139297 19795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [sys.catalog]: This master's current role is: LEADER
Jun 20 09:24:25 dist-test-slave-414w krb5kdc[19707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:24:25.143741 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.132840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46936 (local address 127.4.59.62:35489)
0620 09:24:25.133067 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:25.133076 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:25.133111 (+ 35us) server_negotiation.cc:407] Connection header received
0620 09:24:25.133242 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:25.133248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:25.133302 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:25.133441 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:25.134179 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.135643 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.136729 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.137287 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.137399 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.140195 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:25.140199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:25.140199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:25.140210 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:25.142100 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:25.142516 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:25.142518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:25.142519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:25.142632 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:25.142869 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:25.142870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:25.142870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:25.143149 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:25.143272 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:25.143427 (+ 155us) server_negotiation.cc:299] Negotiation successful
0620 09:24:25.143553 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":113,"threads_started":1}
I20260620 09:24:25.158897 19801 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:24:25.159456 19801 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:24:25.184358 19801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Generated new TSK 0
I20260620 09:24:25.184581 19801 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:24:25.210260 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35489
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:24:25.290344 19814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:25.290530 19814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:25.290552 19814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:25.291581 19814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:25.291613 19814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:25.291638 19814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:24:25.292958 19814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:25.293198 19814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:25.293411 19814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:25.298058 19820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:25.298120 19819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:25.298065 19822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:25.298094 19814 server_base.cc:1034] running on GCE node
I20260620 09:24:25.298401 19814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:25.298664 19814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:25.299835 19814 hybrid_clock.cc:648] HybridClock initialized: now 1781947465299820 us; error 26 us; skew 500 ppm
Jun 20 09:24:25 dist-test-slave-414w krb5kdc[19707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947465, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:25.302346 19814 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:24:25.302784 19814 webserver.cc:460] Webserver started at http://127.4.59.1:46673/ using document root <none> and password file <none>
I20260620 09:24:25.303012 19814 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:25.303077 19814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:25.303231 19814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:24:25.304261 19814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "c3ee7e86e21548b599c884a61dbb8cf7"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "08d28a5b47c6bc799f88e924e0e01c07"
server_key_iv: "5756359c24b981f200a195d032719fed"
server_key_version: "encryptionkey@0"
I20260620 09:24:25.304594 19814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "c3ee7e86e21548b599c884a61dbb8cf7"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "08d28a5b47c6bc799f88e924e0e01c07"
server_key_iv: "5756359c24b981f200a195d032719fed"
server_key_version: "encryptionkey@0"
I20260620 09:24:25.305949 19814 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:24:25.306720 19829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:25.306898 19814 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:24:25.306993 19814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "c3ee7e86e21548b599c884a61dbb8cf7"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "08d28a5b47c6bc799f88e924e0e01c07"
server_key_iv: "5756359c24b981f200a195d032719fed"
server_key_version: "encryptionkey@0"
I20260620 09:24:25.307071 19814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:25.325387 19814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:25.327263 19814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:25.327440 19814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:25.327677 19814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:24:25.327988 19814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:24:25.328044 19814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:25.328078 19814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:24:25.328105 19814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:25.334080 19814 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:33081
I20260620 09:24:25.334127 19942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:33081 every 8 connection(s)
I20260620 09:24:25.334527 19814 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:24:25.334652 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 19814
I20260620 09:24:25.334774 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:24:25.335004 4332 external_mini_cluster.cc:1428] Setting key 22f8a0716dec9653b5a2c30ecaca362d
Jun 20 09:24:25 dist-test-slave-414w krb5kdc[19707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947465, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
I20260620 09:24:25.345446 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.335311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:45801 (local address 127.4.59.62:35489)
0620 09:24:25.335362 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:25.335365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:25.335620 (+ 255us) server_negotiation.cc:407] Connection header received
0620 09:24:25.336372 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:25.336377 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:25.336391 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:25.336454 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:25.337684 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.338450 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.340015 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.340226 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.340283 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.342713 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:25.342714 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:25.342715 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:25.342723 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:25.344102 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:25.344519 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:25.344521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:25.344521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:25.344583 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:25.344817 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:25.344818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:25.344818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:25.344918 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:25.344980 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:25.345248 (+ 268us) server_negotiation.cc:299] Negotiation successful
0620 09:24:25.345315 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 09:24:25.345523 19945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.335311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:45801)
0620 09:24:25.335494 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:25.335540 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:25.336241 (+ 701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:25.336527 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:25.336531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:25.337360 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:25.337590 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:25.337596 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.338539 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.338540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:25.339926 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:25.339929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.340318 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.340320 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:25.340484 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.341001 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:25.341012 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:25.342617 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:25.344203 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:25.344210 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:25.344212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:25.344427 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:25.344654 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:25.344655 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:25.344656 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:25.344754 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:25.344985 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:25.344990 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:25.345096 (+ 106us) client_negotiation.cc:769] Sending connection context
0620 09:24:25.345230 (+ 134us) client_negotiation.cc:240] Negotiation successful
0620 09:24:25.345351 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
I20260620 09:24:25.346038 19943 heartbeater.cc:347] Connected to a master server at 127.4.59.62:35489
I20260620 09:24:25.346133 19943 heartbeater.cc:464] Registering TS with master...
I20260620 09:24:25.346334 19943 heartbeater.cc:511] Master 127.4.59.62:35489 requested a full tablet report, sending...
I20260620 09:24:25.346906 19752 ts_manager.cc:194] Registered new tserver with Master: c3ee7e86e21548b599c884a61dbb8cf7 (127.4.59.1:33081)
I20260620 09:24:25.347793 19752 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:45801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:24:25.360239 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35489
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:24:25.439543 19948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:25.439783 19948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:25.439821 19948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:25.440976 19948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:25.441042 19948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:25.441089 19948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:24:25.442610 19948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:25.442878 19948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:25.443106 19948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:25.448252 19954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:25.448312 19953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:25.448318 19956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:25.448769 19948 server_base.cc:1034] running on GCE node
I20260620 09:24:25.448973 19948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:25.449244 19948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:25.450451 19948 hybrid_clock.cc:648] HybridClock initialized: now 1781947465450433 us; error 45 us; skew 500 ppm
Jun 20 09:24:25 dist-test-slave-414w krb5kdc[19707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947465, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:25.453079 19948 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:24:25.453576 19948 webserver.cc:460] Webserver started at http://127.4.59.2:40709/ using document root <none> and password file <none>
I20260620 09:24:25.453815 19948 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:25.453883 19948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:25.454035 19948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:24:25.455117 19948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "7c3a0cce19dd431f8f5aa0e6624d577b"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "01495d9d13a8f984c9f2e6c77e889514"
server_key_iv: "66ce2fdf78da01c3de8578b465ca096b"
server_key_version: "encryptionkey@0"
I20260620 09:24:25.455479 19948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "7c3a0cce19dd431f8f5aa0e6624d577b"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "01495d9d13a8f984c9f2e6c77e889514"
server_key_iv: "66ce2fdf78da01c3de8578b465ca096b"
server_key_version: "encryptionkey@0"
I20260620 09:24:25.456808 19948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 09:24:25.457652 19963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:25.457847 19948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 09:24:25.457970 19948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "7c3a0cce19dd431f8f5aa0e6624d577b"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "01495d9d13a8f984c9f2e6c77e889514"
server_key_iv: "66ce2fdf78da01c3de8578b465ca096b"
server_key_version: "encryptionkey@0"
I20260620 09:24:25.458045 19948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:25.480831 19948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:25.482748 19948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:25.482929 19948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:25.483172 19948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:24:25.483497 19948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:24:25.483559 19948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:25.483605 19948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:24:25.483632 19948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:25.489581 19948 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:43065
I20260620 09:24:25.489631 20076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:43065 every 8 connection(s)
I20260620 09:24:25.489938 19948 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:24:25.494935 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 19948
I20260620 09:24:25.495012 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:24:25.495252 4332 external_mini_cluster.cc:1428] Setting key 2b6377b73982d3aee3d8cced54a2bf3e
Jun 20 09:24:25 dist-test-slave-414w krb5kdc[19707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947465, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:24:25.499866 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.490663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:42633 (local address 127.4.59.62:35489)
0620 09:24:25.490751 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:25.490753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:25.490925 (+ 172us) server_negotiation.cc:407] Connection header received
0620 09:24:25.491474 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:25.491475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:25.491488 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:25.491533 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:25.492436 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.492947 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.494109 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.494317 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.494409 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.496814 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:25.496816 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:25.496816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:25.496824 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:25.498275 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:25.498799 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:25.498800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:25.498800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:25.498857 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:25.499189 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:25.499191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:25.499191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:25.499307 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:25.499376 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:25.499730 (+ 354us) server_negotiation.cc:299] Negotiation successful
0620 09:24:25.499803 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 09:24:25.500121 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.490621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:42633)
0620 09:24:25.490799 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:25.490845 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:25.491361 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:25.491600 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:25.491604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:25.492146 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:25.492338 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:25.492343 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.493044 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.493045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:25.494008 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:25.494010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.494419 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.494420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:25.494555 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.495123 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:25.495134 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:25.496723 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:25.498357 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:25.498363 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:25.498365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:25.498696 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:25.498932 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:25.498933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:25.498934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:25.499072 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:25.499387 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:25.499393 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:25.499555 (+ 162us) client_negotiation.cc:769] Sending connection context
0620 09:24:25.499679 (+ 124us) client_negotiation.cc:240] Negotiation successful
0620 09:24:25.499856 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260620 09:24:25.500667 20077 heartbeater.cc:347] Connected to a master server at 127.4.59.62:35489
I20260620 09:24:25.500762 20077 heartbeater.cc:464] Registering TS with master...
I20260620 09:24:25.500952 20077 heartbeater.cc:511] Master 127.4.59.62:35489 requested a full tablet report, sending...
I20260620 09:24:25.501351 19752 ts_manager.cc:194] Registered new tserver with Master: 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2:43065)
I20260620 09:24:25.501681 19752 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:42633
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:24:25.520076 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35489
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:24:25.598450 20082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:25.598687 20082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:25.598726 20082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:25.599864 20082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:25.599929 20082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:25.599974 20082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:24:25.601433 20082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:25.601687 20082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:25.601922 20082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 09:24:25.607300 20082 server_base.cc:1034] running on GCE node
W20260620 09:24:25.607250 20088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:25.607270 20087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:25.607244 20090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:25.607811 20082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:25.608018 20082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:25.609197 20082 hybrid_clock.cc:648] HybridClock initialized: now 1781947465609186 us; error 31 us; skew 500 ppm
Jun 20 09:24:25 dist-test-slave-414w krb5kdc[19707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947465, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:25.611835 20082 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:24:25.612246 20082 webserver.cc:460] Webserver started at http://127.4.59.3:46801/ using document root <none> and password file <none>
I20260620 09:24:25.612493 20082 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:25.612560 20082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:25.612706 20082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:24:25.613807 20082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "609c46105ce04dc487af086a83f105a4"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "9d7bd2cb1e7b6c510c7d101d1faacd08"
server_key_iv: "6c9da144fb0198ab9ddac967e74d0a12"
server_key_version: "encryptionkey@0"
I20260620 09:24:25.614161 20082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "609c46105ce04dc487af086a83f105a4"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "9d7bd2cb1e7b6c510c7d101d1faacd08"
server_key_iv: "6c9da144fb0198ab9ddac967e74d0a12"
server_key_version: "encryptionkey@0"
I20260620 09:24:25.615615 20082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:24:25.616523 20097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:25.616735 20082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:25.616835 20082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "609c46105ce04dc487af086a83f105a4"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "9d7bd2cb1e7b6c510c7d101d1faacd08"
server_key_iv: "6c9da144fb0198ab9ddac967e74d0a12"
server_key_version: "encryptionkey@0"
I20260620 09:24:25.616914 20082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:25.634568 20082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:25.637101 20082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:25.637277 20082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:25.637523 20082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:24:25.637838 20082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:24:25.637889 20082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:25.637929 20082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:24:25.637957 20082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:25.643688 20082 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:46521
I20260620 09:24:25.643750 20210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:46521 every 8 connection(s)
I20260620 09:24:25.644047 20082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:24:25.644474 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 20082
I20260620 09:24:25.644569 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:24:25.644771 4332 external_mini_cluster.cc:1428] Setting key b751f8e13451467b26573a373580e722
Jun 20 09:24:25 dist-test-slave-414w krb5kdc[19707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947465, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:24:25.653402 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.644935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:57337 (local address 127.4.59.62:35489)
0620 09:24:25.645002 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:25.645004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:25.645260 (+ 256us) server_negotiation.cc:407] Connection header received
0620 09:24:25.645887 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:25.645888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:25.645900 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:25.645970 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:25.646937 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.647439 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.648450 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.648646 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.648737 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.650800 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:25.650801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:25.650802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:25.650808 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:25.652154 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:25.652539 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:25.652540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:25.652540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:25.652589 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:25.652852 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:25.652852 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:25.652853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:25.652948 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:25.652997 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:25.653271 (+ 274us) server_negotiation.cc:299] Negotiation successful
0620 09:24:25.653341 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 09:24:25.653647 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.644935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:57337)
0620 09:24:25.645109 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:25.645180 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:25.645773 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:25.646042 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:25.646046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:25.646630 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:25.646824 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:25.646828 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.647535 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.647536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:25.648339 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:25.648342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.648746 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.648747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:25.648869 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.649401 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:25.649409 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:25.650710 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:25.652251 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:25.652257 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:25.652258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:25.652471 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:25.652676 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:25.652677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:25.652677 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:25.652784 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:25.653006 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:25.653010 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:25.653115 (+ 105us) client_negotiation.cc:769] Sending connection context
0620 09:24:25.653257 (+ 142us) client_negotiation.cc:240] Negotiation successful
0620 09:24:25.653395 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260620 09:24:25.654140 20211 heartbeater.cc:347] Connected to a master server at 127.4.59.62:35489
I20260620 09:24:25.654213 20211 heartbeater.cc:464] Registering TS with master...
I20260620 09:24:25.654345 20211 heartbeater.cc:511] Master 127.4.59.62:35489 requested a full tablet report, sending...
I20260620 09:24:25.654654 19752 ts_manager.cc:194] Registered new tserver with Master: 609c46105ce04dc487af086a83f105a4 (127.4.59.3:46521)
I20260620 09:24:25.654969 19752 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:57337
I20260620 09:24:25.656270 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:24:25.663451 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.657560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46946 (local address 127.4.59.62:35489)
0620 09:24:25.657635 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:25.657637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:25.657752 (+ 115us) server_negotiation.cc:407] Connection header received
0620 09:24:25.657823 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:25.657824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:25.657835 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:25.657888 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:25.658409 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.658852 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.659576 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.659774 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.659845 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.660952 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:25.660954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:25.660954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:25.660961 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:25.662275 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:25.662652 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:25.662653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:25.662653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:25.662705 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:25.662923 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:25.662924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:25.662924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:25.663064 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:25.663169 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:25.663284 (+ 115us) server_negotiation.cc:299] Negotiation successful
0620 09:24:25.663358 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 09:24:25.664862 19752 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:24:25.666496 19752 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:24:25.672709 20219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.668565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:33081 (local address 127.0.0.1:58830)
0620 09:24:25.668707 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:25.668738 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:25.668793 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:25.669131 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:25.669133 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:25.669163 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:25.669226 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:25.669228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.670514 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.670515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:25.671627 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:25.671630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.672415 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.672416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:25.672503 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.672516 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 09:24:25.672590 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 09:24:25.672637 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":4,"thread_start_us":90,"threads_started":1}
I20260620 09:24:25.672741 20220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.668695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58830 (local address 127.4.59.1:33081)
0620 09:24:25.668957 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:25.668961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:25.668988 (+ 27us) server_negotiation.cc:407] Connection header received
0620 09:24:25.669029 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:25.669031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:25.669048 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:25.669126 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:25.669339 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.670401 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.671723 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.672328 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.672404 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.672458 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:25.672618 (+ 160us) server_negotiation.cc:299] Negotiation successful
0620 09:24:25.672660 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":106,"threads_started":1}
I20260620 09:24:25.673206 20221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.668825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:46521 (local address 127.0.0.1:37722)
0620 09:24:25.668915 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:25.668925 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:25.668972 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:25.669572 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:25.669573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:25.669575 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:25.669619 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:25.669621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.670798 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.670799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:25.671958 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:25.671960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.672934 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.672935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:25.673006 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.673010 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:24:25.673071 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 09:24:25.673110 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":34,"thread_start_us":55,"threads_started":1}
I20260620 09:24:25.673270 20223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.669080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37722 (local address 127.4.59.3:46521)
0620 09:24:25.669300 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:25.669304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:25.669314 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:24:25.669444 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:25.669448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:25.669465 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:25.669561 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:25.669706 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.670685 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.672049 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.672770 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.672840 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.672894 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:25.673103 (+ 209us) server_negotiation.cc:299] Negotiation successful
0620 09:24:25.673182 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":107,"threads_started":1}
I20260620 09:24:25.673650 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.668982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:43065 (local address 127.0.0.1:43964)
0620 09:24:25.669668 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:25.669678 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:25.669726 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:25.669933 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:25.669934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:25.669936 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:25.669978 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:25.669979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.671013 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.671014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:25.672098 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:25.672099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.673416 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.673416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:25.673496 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.673501 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 09:24:25.673546 (+ 45us) client_negotiation.cc:240] Negotiation successful
0620 09:24:25.673587 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":39,"threads_started":1}
I20260620 09:24:25.673609 19877 tablet_service.cc:1467] Processing CreateTablet for tablet 57c0143f868d4207bf76338fe1e9ff59 (DEFAULT_TABLE table=test-table [id=dba12231562c49779cc4450cf54dc6e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:24:25.673966 19877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c0143f868d4207bf76338fe1e9ff59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:25.674007 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.669109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43964 (local address 127.4.59.2:43065)
0620 09:24:25.669315 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:25.669319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:25.669784 (+ 465us) server_negotiation.cc:407] Connection header received
0620 09:24:25.669827 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:25.669829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:25.669850 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:25.669935 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:25.670034 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.670908 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.672179 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.673120 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.673248 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.673857 (+ 609us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:25.673884 (+ 27us) server_negotiation.cc:299] Negotiation successful
0620 09:24:25.673926 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":133,"threads_started":1}
I20260620 09:24:25.674062 20145 tablet_service.cc:1467] Processing CreateTablet for tablet 57c0143f868d4207bf76338fe1e9ff59 (DEFAULT_TABLE table=test-table [id=dba12231562c49779cc4450cf54dc6e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:24:25.674364 20145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c0143f868d4207bf76338fe1e9ff59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:25.677367 20225 tablet_bootstrap.cc:492] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7: Bootstrap starting.
I20260620 09:24:25.677699 20226 tablet_bootstrap.cc:492] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4: Bootstrap starting.
I20260620 09:24:25.678005 20225 tablet_bootstrap.cc:654] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:25.677935 20011 tablet_service.cc:1467] Processing CreateTablet for tablet 57c0143f868d4207bf76338fe1e9ff59 (DEFAULT_TABLE table=test-table [id=dba12231562c49779cc4450cf54dc6e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:24:25.678243 20011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c0143f868d4207bf76338fe1e9ff59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:25.678390 20226 tablet_bootstrap.cc:654] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:25.678637 20225 log.cc:824] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:25.678934 20226 log.cc:824] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:25.679414 20225 tablet_bootstrap.cc:492] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7: No bootstrap required, opened a new log
I20260620 09:24:25.679498 20225 ts_tablet_manager.cc:1397] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:24:25.679666 20226 tablet_bootstrap.cc:492] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4: No bootstrap required, opened a new log
I20260620 09:24:25.679745 20226 ts_tablet_manager.cc:1397] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:24:25.681030 20225 raft_consensus.cc:348] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:25.681326 20225 raft_consensus.cc:374] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:25.681396 20225 raft_consensus.cc:729] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3ee7e86e21548b599c884a61dbb8cf7, State: Initialized, Role: FOLLOWER
I20260620 09:24:25.681522 20225 consensus_queue.cc:260] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:25.681829 20225 ts_tablet_manager.cc:1428] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 09:24:25.681859 19943 heartbeater.cc:503] Master 127.4.59.62:35489 was elected leader, sending a full tablet report...
I20260620 09:24:25.681850 20226 raft_consensus.cc:348] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:25.682022 20226 raft_consensus.cc:374] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:25.682057 20226 raft_consensus.cc:729] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 609c46105ce04dc487af086a83f105a4, State: Initialized, Role: FOLLOWER
I20260620 09:24:25.682165 20226 consensus_queue.cc:260] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:25.682449 20226 ts_tablet_manager.cc:1428] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 09:24:25.682469 20211 heartbeater.cc:503] Master 127.4.59.62:35489 was elected leader, sending a full tablet report...
I20260620 09:24:25.682641 20229 tablet_bootstrap.cc:492] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b: Bootstrap starting.
I20260620 09:24:25.683437 20229 tablet_bootstrap.cc:654] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:25.683904 20229 log.cc:824] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:25.684585 20229 tablet_bootstrap.cc:492] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b: No bootstrap required, opened a new log
I20260620 09:24:25.684684 20229 ts_tablet_manager.cc:1397] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:24:25.686096 20229 raft_consensus.cc:348] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:25.686233 20229 raft_consensus.cc:374] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:25.686272 20229 raft_consensus.cc:729] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3a0cce19dd431f8f5aa0e6624d577b, State: Initialized, Role: FOLLOWER
I20260620 09:24:25.686342 20229 consensus_queue.cc:260] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:25.686524 20229 ts_tablet_manager.cc:1428] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:24:25.686609 20077 heartbeater.cc:503] Master 127.4.59.62:35489 was elected leader, sending a full tablet report...
I20260620 09:24:25.690622 20233 raft_consensus.cc:482] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:24:25.690687 20233 raft_consensus.cc:504] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:25.690915 20233 leader_election.cc:288] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3ee7e86e21548b599c884a61dbb8cf7 (127.4.59.1:33081), 609c46105ce04dc487af086a83f105a4 (127.4.59.3:46521)
I20260620 09:24:25.693620 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.691073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:33081 (local address 127.4.59.2:35639)
0620 09:24:25.691139 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:25.691152 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:25.691201 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:25.691344 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:25.691345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:25.691346 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:25.691405 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:25.691407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.692001 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.692002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:25.692716 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:25.692717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.693356 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.693357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:25.693436 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.693444 (+ 8us) client_negotiation.cc:769] Sending connection context
0620 09:24:25.693515 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 09:24:25.693554 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":3}
I20260620 09:24:25.693617 20220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.691144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:35639 (local address 127.4.59.1:33081)
0620 09:24:25.691195 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:25.691197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:25.691206 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:24:25.691273 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:25.691274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:25.691283 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:25.691340 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:25.691467 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.691923 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.692829 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.693267 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.693354 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.693364 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:25.693522 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 09:24:25.693557 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 09:24:25.693925 19897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c0143f868d4207bf76338fe1e9ff59" candidate_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" is_pre_election: true
I20260620 09:24:25.693975 20223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.691317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:40111 (local address 127.4.59.3:46521)
0620 09:24:25.691359 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:25.691361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:25.691370 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:24:25.691425 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:25.691426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:25.691435 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:25.691489 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:25.691632 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.692072 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.692980 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.693449 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.693538 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.693548 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:25.693871 (+ 323us) server_negotiation.cc:299] Negotiation successful
0620 09:24:25.693914 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 09:24:25.693972 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.691167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:46521 (local address 127.4.59.2:40111)
0620 09:24:25.691312 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:25.691322 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:25.691375 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:25.691495 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:25.691495 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:25.691497 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:25.691556 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:25.691558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.692152 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.692153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:25.692882 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:25.692883 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.693655 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.693656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:25.693770 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.693774 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:24:25.693859 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 09:24:25.693894 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260620 09:24:25.694047 19897 raft_consensus.cc:2454] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c3a0cce19dd431f8f5aa0e6624d577b in term 0.
I20260620 09:24:25.694324 19967 leader_election.cc:302] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c3a0cce19dd431f8f5aa0e6624d577b, c3ee7e86e21548b599c884a61dbb8cf7; no voters:
I20260620 09:24:25.694432 20233 raft_consensus.cc:2781] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:24:25.694490 20233 raft_consensus.cc:482] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:24:25.694514 20233 raft_consensus.cc:3037] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:25.694661 20165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c0143f868d4207bf76338fe1e9ff59" candidate_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "609c46105ce04dc487af086a83f105a4" is_pre_election: true
I20260620 09:24:25.694793 20165 raft_consensus.cc:2454] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c3a0cce19dd431f8f5aa0e6624d577b in term 0.
I20260620 09:24:25.695286 20233 raft_consensus.cc:504] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:25.695406 20233 leader_election.cc:288] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [CANDIDATE]: Term 1 election: Requested vote from peers c3ee7e86e21548b599c884a61dbb8cf7 (127.4.59.1:33081), 609c46105ce04dc487af086a83f105a4 (127.4.59.3:46521)
I20260620 09:24:25.695567 20165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c0143f868d4207bf76338fe1e9ff59" candidate_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "609c46105ce04dc487af086a83f105a4"
I20260620 09:24:25.695582 19897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c0143f868d4207bf76338fe1e9ff59" candidate_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3ee7e86e21548b599c884a61dbb8cf7"
I20260620 09:24:25.695663 20165 raft_consensus.cc:3037] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:25.695660 19897 raft_consensus.cc:3037] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:25.696333 20165 raft_consensus.cc:2454] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c3a0cce19dd431f8f5aa0e6624d577b in term 1.
I20260620 09:24:25.696354 19897 raft_consensus.cc:2454] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c3a0cce19dd431f8f5aa0e6624d577b in term 1.
I20260620 09:24:25.696520 19967 leader_election.cc:302] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c3a0cce19dd431f8f5aa0e6624d577b, c3ee7e86e21548b599c884a61dbb8cf7; no voters:
I20260620 09:24:25.696633 20233 raft_consensus.cc:2781] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:24:25.696794 20233 raft_consensus.cc:686] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [term 1 LEADER]: Becoming Leader. State: Replica: 7c3a0cce19dd431f8f5aa0e6624d577b, State: Running, Role: LEADER
I20260620 09:24:25.696897 20233 consensus_queue.cc:237] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:25.697665 19751 catalog_manager.cc:5495] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2). New cstate: current_term: 1 leader_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } health_report { overall_health: HEALTHY } } }
I20260620 09:24:25.704710 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.702358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43978 (local address 127.4.59.2:43065)
0620 09:24:25.702410 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:25.702412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:25.702419 (+ 7us) server_negotiation.cc:407] Connection header received
0620 09:24:25.702446 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:25.702447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:25.702468 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:25.702544 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:24:25.702686 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.703129 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.703914 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:25.704119 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:25.704192 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:25.704371 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:24:25.704471 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:24:25.704555 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:25.704596 (+ 41us) server_negotiation.cc:299] Negotiation successful
0620 09:24:25.704614 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 09:24:25.706352 19897 raft_consensus.cc:1261] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Refusing update from remote peer 7c3a0cce19dd431f8f5aa0e6624d577b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:24:25.706382 20165 raft_consensus.cc:1261] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4 [term 1 FOLLOWER]: Refusing update from remote peer 7c3a0cce19dd431f8f5aa0e6624d577b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:24:25.706728 20233 consensus_queue.cc:1035] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [LEADER]: Connected to new peer: Peer: permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:24:25.706842 20235 consensus_queue.cc:1035] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:24:25.709129 20238 mvcc.cc:205] Tried to move back new op lower bound from 7298856819531149312 to 7298856819495501824. Current Snapshot: MvccSnapshot[applied={T|T < 7298856819531149312}]
I20260620 09:24:25.709646 20240 mvcc.cc:205] Tried to move back new op lower bound from 7298856819531149312 to 7298856819495501824. Current Snapshot: MvccSnapshot[applied={T|T < 7298856819531149312}]
I20260620 09:24:25.709656 20239 mvcc.cc:205] Tried to move back new op lower bound from 7298856819531149312 to 7298856819495501824. Current Snapshot: MvccSnapshot[applied={T|T < 7298856819531149312}]
I20260620 09:24:25.711748 19752 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:24:25.711834 19752 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:24:25.712538 19752 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Sending DeleteTablet for 3 replicas of tablet 57c0143f868d4207bf76338fe1e9ff59
I20260620 09:24:25.712839 19877 tablet_service.cc:1514] Processing DeleteTablet for tablet 57c0143f868d4207bf76338fe1e9ff59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:25 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:58830
I20260620 09:24:25.712894 20145 tablet_service.cc:1514] Processing DeleteTablet for tablet 57c0143f868d4207bf76338fe1e9ff59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:25 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:37722
I20260620 09:24:25.712941 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 19814
I20260620 09:24:25.714301 20011 tablet_service.cc:1514] Processing DeleteTablet for tablet 57c0143f868d4207bf76338fe1e9ff59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:25 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:43964
W20260620 09:24:25.717399 19739 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.59.1:33081 (error 108)
I20260620 09:24:25.717917 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.717761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:33081 (local address 127.0.0.1:58832)
0620 09:24:25.717835 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:25.717879 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:33081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 09:24:25.717928 20249 tablet_replica.cc:330] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4: stopping tablet replica
I20260620 09:24:25.718045 20249 raft_consensus.cc:2229] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260620 09:24:25.718128 19739 catalog_manager.cc:4584] TS c3ee7e86e21548b599c884a61dbb8cf7 (127.4.59.1:33081): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 57c0143f868d4207bf76338fe1e9ff59: Network error: Client connection negotiation failed: client connection to 127.4.59.1:33081: connect: Connection refused (error 111)
I20260620 09:24:25.718156 20249 raft_consensus.cc:2258] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:24:25.718356 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 19948
I20260620 09:24:25.719444 20249 ts_tablet_manager.cc:1905] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:24:25.721050 20249 ts_tablet_manager.cc:1918] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:24:25.721122 20249 log.cc:1196] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/57c0143f868d4207bf76338fe1e9ff59
I20260620 09:24:25.721433 20249 ts_tablet_manager.cc:1939] T 57c0143f868d4207bf76338fe1e9ff59 P 609c46105ce04dc487af086a83f105a4: Deleting consensus metadata
I20260620 09:24:25.721822 19739 catalog_manager.cc:4841] TS 609c46105ce04dc487af086a83f105a4 (127.4.59.3:46521): tablet 57c0143f868d4207bf76338fe1e9ff59 (table test-table [id=dba12231562c49779cc4450cf54dc6e5]) successfully deleted
I20260620 09:24:25.724330 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:25.724203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:43065 (local address 127.0.0.1:43986)
0620 09:24:25.724256 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:25.724285 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.2:43065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 09:24:25.724473 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 20082
I20260620 09:24:25.728736 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 19720
I20260620 09:24:45.734193 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:35489
--webserver_interface=127.4.59.62
--webserver_port=43619
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:35489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:24:45.818789 20252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:45.818969 20252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:45.818995 20252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:45.820118 20252 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 09:24:45.820158 20252 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 09:24:45.820171 20252 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 09:24:45.820182 20252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:24:45.820220 20252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:45.820248 20252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:45.820264 20252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:24:45.820288 20252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:24:45.821796 20252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:35489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:35489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=43619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:45.822037 20252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:45.822257 20252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:45.830528 20261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:45.830550 20259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:45.830763 20252 server_base.cc:1034] running on GCE node
W20260620 09:24:45.830607 20258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:45.831110 20252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:45.831422 20252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:45.832584 20252 hybrid_clock.cc:648] HybridClock initialized: now 1781947485832574 us; error 44 us; skew 500 ppm
Jun 20 09:24:45 dist-test-slave-414w krb5kdc[19707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947485, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:45.839570 20252 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:24:45.840005 20252 webserver.cc:460] Webserver started at http://127.4.59.62:43619/ using document root <none> and password file <none>
I20260620 09:24:45.840238 20252 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:45.840317 20252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:45.841769 20252 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:24:45.842453 20268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:45.842669 20252 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:45.842774 20252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "44493f35a1514b2ba46b0e3d05665518"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "faae8e09901bf012f1d3610bc33f4470"
server_key_iv: "625ffa47380b5158e9470664a05d3a85"
server_key_version: "encryptionkey@0"
I20260620 09:24:45.843180 20252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:45.855013 20252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:45.857385 20252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:45.857532 20252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:45.861377 20252 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:35489
I20260620 09:24:45.861454 20320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:35489 every 8 connection(s)
I20260620 09:24:45.861814 20252 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:24:45.863245 20321 sys_catalog.cc:263] Verifying existing consensus state
I20260620 09:24:45.863901 20321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Bootstrap starting.
I20260620 09:24:45.869824 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 20252
I20260620 09:24:45.870136 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:33081
--local_ip_for_outbound_sockets=127.4.59.1
--tserver_master_addrs=127.4.59.62:35489
--webserver_port=46673
--webserver_interface=127.4.59.1
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:24:45.872597 20321 log.cc:824] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:45.874528 20321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 09:24:45.874799 20321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Bootstrap complete.
I20260620 09:24:45.877041 20321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } }
I20260620 09:24:45.877331 20321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44493f35a1514b2ba46b0e3d05665518, State: Initialized, Role: FOLLOWER
I20260620 09:24:45.877527 20321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } }
I20260620 09:24:45.877620 20321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:24:45.877676 20321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:24:45.877723 20321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 1 FOLLOWER]: Advancing to term 2
I20260620 09:24:45.878679 20321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } }
I20260620 09:24:45.878795 20321 leader_election.cc:302] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44493f35a1514b2ba46b0e3d05665518; no voters:
I20260620 09:24:45.878993 20321 leader_election.cc:288] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 09:24:45.879025 20325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 09:24:45.879137 20325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 2 LEADER]: Becoming Leader. State: Replica: 44493f35a1514b2ba46b0e3d05665518, State: Running, Role: LEADER
I20260620 09:24:45.879248 20321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:24:45.879290 20325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } }
I20260620 09:24:45.879899 20327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44493f35a1514b2ba46b0e3d05665518. Latest consensus state: current_term: 2 leader_uuid: "44493f35a1514b2ba46b0e3d05665518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } } }
I20260620 09:24:45.880002 20327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [sys.catalog]: This master's current role is: LEADER
I20260620 09:24:45.880218 20326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "44493f35a1514b2ba46b0e3d05665518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } } }
I20260620 09:24:45.880291 20326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [sys.catalog]: This master's current role is: LEADER
I20260620 09:24:45.880471 20334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:24:45.881253 20334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:24:45.881397 20334 catalog_manager.cc:1259] Loaded cluster ID: a588143cad7c4a15bdbabdb94789f318
I20260620 09:24:45.881445 20334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:24:45.882241 20334 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:24:45.882417 20334 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Loaded TSK: 0
I20260620 09:24:45.889993 20334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Generated new TSK 1
I20260620 09:24:45.890084 20334 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 09:24:45.967962 20323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:45.968174 20323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:45.968209 20323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:45.969323 20323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:45.969385 20323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:45.969440 20323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:24:45.970845 20323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:33081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=46673
--tserver_master_addrs=127.4.59.62:35489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:45.971091 20323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:45.971298 20323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:45.977008 20349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:45.977020 20351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:45.977169 20323 server_base.cc:1034] running on GCE node
W20260620 09:24:45.977003 20348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:45.977481 20323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:45.977756 20323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:45.978943 20323 hybrid_clock.cc:648] HybridClock initialized: now 1781947485978925 us; error 33 us; skew 500 ppm
Jun 20 09:24:45 dist-test-slave-414w krb5kdc[19707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947485, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:45.981910 20323 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:24:45.982426 20323 webserver.cc:460] Webserver started at http://127.4.59.1:46673/ using document root <none> and password file <none>
I20260620 09:24:45.982671 20323 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:45.982753 20323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:45.984424 20323 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:45.985209 20358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:45.985433 20323 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:45.985527 20323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "c3ee7e86e21548b599c884a61dbb8cf7"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "08d28a5b47c6bc799f88e924e0e01c07"
server_key_iv: "5756359c24b981f200a195d032719fed"
server_key_version: "encryptionkey@0"
I20260620 09:24:45.985939 20323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:46.008850 20323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:46.011466 20323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:46.011632 20323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:46.011902 20323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:24:46.012398 20365 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 09:24:46.013947 20323 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 09:24:46.014007 20323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 09:24:46.014042 20323 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 09:24:46.014873 20323 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 09:24:46.014918 20323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 09:24:46.015004 20365 tablet_bootstrap.cc:492] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7: Bootstrap starting.
I20260620 09:24:46.021600 20323 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:33081
I20260620 09:24:46.021778 20472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:33081 every 8 connection(s)
I20260620 09:24:46.022053 20323 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:24:46.025653 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 20323
I20260620 09:24:46.025930 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:43065
--local_ip_for_outbound_sockets=127.4.59.2
--tserver_master_addrs=127.4.59.62:35489
--webserver_port=40709
--webserver_interface=127.4.59.2
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:24:46.027313 20365 log.cc:824] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:46.030088 20365 tablet_bootstrap.cc:492] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 09:24:46.030416 20365 tablet_bootstrap.cc:492] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7: Bootstrap complete.
I20260620 09:24:46.031628 20365 ts_tablet_manager.cc:1397] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.007s
Jun 20 09:24:46 dist-test-slave-414w krb5kdc[19707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947485, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:24:46.033281 20365 raft_consensus.cc:348] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:46.034263 20365 raft_consensus.cc:729] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3ee7e86e21548b599c884a61dbb8cf7, State: Initialized, Role: FOLLOWER
I20260620 09:24:46.034524 20365 consensus_queue.cc:260] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:46.034888 20365 ts_tablet_manager.cc:1428] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 09:24:46.036700 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.022768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:41773)
0620 09:24:46.023092 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:46.023136 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:46.023935 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:46.024803 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:46.024813 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:46.025670 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:46.026004 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:46.026014 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.027864 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.027866 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:46.029107 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:46.029111 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.030054 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.030055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:46.030204 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.030828 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:46.030848 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:46.032639 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:46.034894 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:46.034900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:46.034902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:46.035186 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:46.035501 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:46.035502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:46.035502 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:46.035623 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:46.036102 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:46.036107 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:46.036245 (+ 138us) client_negotiation.cc:769] Sending connection context
0620 09:24:46.036353 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 09:24:46.036508 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":110,"threads_started":1}
I20260620 09:24:46.036826 20476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.022853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:41773 (local address 127.4.59.62:35489)
0620 09:24:46.023028 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:46.023037 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:46.023254 (+ 217us) server_negotiation.cc:407] Connection header received
0620 09:24:46.024168 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:46.024175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:46.024230 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:46.024380 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:46.026192 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.027713 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.029265 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.029937 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.030049 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.032752 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:46.032757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:46.032757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:46.032768 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:46.034798 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:46.035279 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:46.035283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:46.035283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:46.035411 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:46.035700 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:46.035702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:46.035702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:46.036023 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:46.036122 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:46.036432 (+ 310us) server_negotiation.cc:299] Negotiation successful
0620 09:24:46.036596 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
I20260620 09:24:46.037734 20473 heartbeater.cc:347] Connected to a master server at 127.4.59.62:35489
I20260620 09:24:46.037829 20473 heartbeater.cc:464] Registering TS with master...
I20260620 09:24:46.038012 20473 heartbeater.cc:511] Master 127.4.59.62:35489 requested a full tablet report, sending...
I20260620 09:24:46.038795 20285 ts_manager.cc:194] Registered new tserver with Master: c3ee7e86e21548b599c884a61dbb8cf7 (127.4.59.1:33081)
I20260620 09:24:46.040055 20285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:41773
I20260620 09:24:46.041065 20473 heartbeater.cc:503] Master 127.4.59.62:35489 was elected leader, sending a full tablet report...
I20260620 09:24:46.044682 20481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.040359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43042 (local address 127.4.59.1:33081)
0620 09:24:46.040514 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:46.040523 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:46.041773 (+ 1250us) server_negotiation.cc:407] Connection header received
0620 09:24:46.041873 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:46.041875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:46.041891 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:46.041953 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:46.042091 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.042762 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.043823 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.044278 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.044375 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.044429 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:46.044582 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 09:24:46.044618 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260620 09:24:46.044684 20480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.039442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:33081 (local address 127.0.0.1:43042)
0620 09:24:46.041673 (+ 2231us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:46.041698 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:46.041797 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:46.041965 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:46.041968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:46.041970 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:46.042028 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:46.042030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.042861 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.042862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:46.043719 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:46.043721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.044376 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.044377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:46.044457 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.044470 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 09:24:46.044565 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 09:24:46.044617 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2212,"thread_start_us":85,"threads_started":1}
I20260620 09:24:46.045049 20407 tablet_service.cc:1514] Processing DeleteTablet for tablet 57c0143f868d4207bf76338fe1e9ff59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:25 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:43042
I20260620 09:24:46.045619 20482 tablet_replica.cc:330] stopping tablet replica
I20260620 09:24:46.045734 20482 raft_consensus.cc:2229] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:24:46.045819 20482 raft_consensus.cc:2258] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:24:46.046152 20482 ts_tablet_manager.cc:1905] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:24:46.048426 20482 ts_tablet_manager.cc:1918] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:24:46.048514 20482 log.cc:1196] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal/wals/57c0143f868d4207bf76338fe1e9ff59
I20260620 09:24:46.048733 20482 ts_tablet_manager.cc:1939] T 57c0143f868d4207bf76338fe1e9ff59 P c3ee7e86e21548b599c884a61dbb8cf7: Deleting consensus metadata
I20260620 09:24:46.049208 20272 catalog_manager.cc:4841] TS c3ee7e86e21548b599c884a61dbb8cf7 (127.4.59.1:33081): tablet 57c0143f868d4207bf76338fe1e9ff59 (table test-table [id=dba12231562c49779cc4450cf54dc6e5]) successfully deleted
W20260620 09:24:46.112411 20477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:46.112607 20477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:46.112629 20477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:46.113682 20477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:46.113715 20477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:46.113740 20477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:24:46.115069 20477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:43065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=40709
--tserver_master_addrs=127.4.59.62:35489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:46.115234 20477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:46.115391 20477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:46.120149 20488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:46.120155 20487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:46.120250 20490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:46.120390 20477 server_base.cc:1034] running on GCE node
I20260620 09:24:46.120580 20477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:46.120761 20477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:46.121927 20477 hybrid_clock.cc:648] HybridClock initialized: now 1781947486121914 us; error 30 us; skew 500 ppm
Jun 20 09:24:46 dist-test-slave-414w krb5kdc[19707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947486, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:46.124192 20477 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:24:46.124612 20477 webserver.cc:460] Webserver started at http://127.4.59.2:40709/ using document root <none> and password file <none>
I20260620 09:24:46.124828 20477 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:46.124908 20477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:46.126216 20477 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:46.126848 20497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:46.127014 20477 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:24:46.127110 20477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "7c3a0cce19dd431f8f5aa0e6624d577b"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "01495d9d13a8f984c9f2e6c77e889514"
server_key_iv: "66ce2fdf78da01c3de8578b465ca096b"
server_key_version: "encryptionkey@0"
I20260620 09:24:46.127422 20477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:46.144459 20477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:46.146402 20477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:46.146560 20477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:46.146775 20477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:24:46.147233 20504 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 09:24:46.148630 20477 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 09:24:46.148706 20477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 09:24:46.148752 20477 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 09:24:46.149433 20477 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 09:24:46.149492 20477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:46.149544 20504 tablet_bootstrap.cc:492] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b: Bootstrap starting.
I20260620 09:24:46.155997 20477 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:43065
I20260620 09:24:46.156072 20611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:43065 every 8 connection(s)
I20260620 09:24:46.156363 20477 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:24:46.160089 20504 log.cc:824] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:46.162668 20504 tablet_bootstrap.cc:492] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 09:24:46.163007 20504 tablet_bootstrap.cc:492] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b: Bootstrap complete.
Jun 20 09:24:46 dist-test-slave-414w krb5kdc[19707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947486, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:24:46.163944 20504 ts_tablet_manager.cc:1397] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.006s
I20260620 09:24:46.165032 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 20477
I20260620 09:24:46.164819 20504 raft_consensus.cc:348] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:46.165228 20504 raft_consensus.cc:729] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3a0cce19dd431f8f5aa0e6624d577b, State: Initialized, Role: FOLLOWER
I20260620 09:24:46.165346 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:46521
--local_ip_for_outbound_sockets=127.4.59.3
--tserver_master_addrs=127.4.59.62:35489
--webserver_port=46801
--webserver_interface=127.4.59.3
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:24:46.165392 20504 consensus_queue.cc:260] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:46.165666 20504 ts_tablet_manager.cc:1428] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260620 09:24:46.167084 20476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.157186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:50801 (local address 127.4.59.62:35489)
0620 09:24:46.157246 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:46.157248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:46.157497 (+ 249us) server_negotiation.cc:407] Connection header received
0620 09:24:46.158314 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:46.158315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:46.158325 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:46.158375 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:46.159317 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.159888 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.161302 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.161508 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.161566 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.163927 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:46.163928 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:46.163929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:46.163938 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:46.165601 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:46.166084 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:46.166085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:46.166086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:46.166145 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:46.166414 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:46.166415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:46.166416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:46.166534 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:46.166612 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:46.166870 (+ 258us) server_negotiation.cc:299] Negotiation successful
0620 09:24:46.166985 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 09:24:46.167197 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.157114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:50801)
0620 09:24:46.157319 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:46.157379 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:46.158180 (+ 801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:46.158461 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:46.158466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:46.158890 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:46.159203 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:46.159211 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.160013 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.160014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:46.161216 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:46.161219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.161620 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.161621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:46.161763 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.162272 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:46.162282 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:46.163810 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:46.165691 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:46.165696 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:46.165697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:46.165999 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:46.166221 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:46.166222 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:46.166222 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:46.166340 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:46.166602 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:46.166608 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:46.166744 (+ 136us) client_negotiation.cc:769] Sending connection context
0620 09:24:46.166849 (+ 105us) client_negotiation.cc:240] Negotiation successful
0620 09:24:46.167004 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":115,"threads_started":1}
I20260620 09:24:46.167711 20612 heartbeater.cc:347] Connected to a master server at 127.4.59.62:35489
I20260620 09:24:46.167809 20612 heartbeater.cc:464] Registering TS with master...
I20260620 09:24:46.167986 20612 heartbeater.cc:511] Master 127.4.59.62:35489 requested a full tablet report, sending...
I20260620 09:24:46.168545 20285 ts_manager.cc:194] Registered new tserver with Master: 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2:43065)
I20260620 09:24:46.169164 20285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:50801
I20260620 09:24:46.170575 20612 heartbeater.cc:503] Master 127.4.59.62:35489 was elected leader, sending a full tablet report...
Jun 20 09:24:46 dist-test-slave-414w krb5kdc[19707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947485, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for kudu/127.4.59.2@KRBTEST.COM
I20260620 09:24:46.176275 20480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.168855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:43065 (local address 127.0.0.1:40544)
0620 09:24:46.168909 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:46.168918 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:46.168958 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:46.169209 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:46.169210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:46.169616 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:46.169664 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:46.169666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.170484 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.170485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:46.171189 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:46.171192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.171666 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.171669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:46.171725 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.172228 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:46.172232 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:46.173481 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:46.175135 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:46.175143 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:46.175144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:46.175342 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:46.175585 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:46.175586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:46.175586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:46.175652 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:46.175926 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:46.175933 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:46.175983 (+ 50us) client_negotiation.cc:769] Sending connection context
0620 09:24:46.176075 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 09:24:46.176179 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 09:24:46.176277 20618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.168919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.4.59.2:43065)
0620 09:24:46.169043 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:46.169047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:46.169056 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:24:46.169086 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:46.169088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:46.169102 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:46.169208 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:46.169749 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.170396 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.171280 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.171579 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.171665 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.173585 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:46.173592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:46.173592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:46.173598 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:46.175035 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:46.175444 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:46.175446 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:46.175447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:46.175506 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:46.175713 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:46.175715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:46.175715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:46.175854 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:46.175944 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:46.176110 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 09:24:46.176195 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260620 09:24:46.176638 20546 tablet_service.cc:1514] Processing DeleteTablet for tablet 57c0143f868d4207bf76338fe1e9ff59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:25 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:40544
I20260620 09:24:46.177119 20619 tablet_replica.cc:330] stopping tablet replica
I20260620 09:24:46.177240 20619 raft_consensus.cc:2229] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:24:46.177306 20619 raft_consensus.cc:2258] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:24:46.177583 20619 ts_tablet_manager.cc:1905] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:24:46.179759 20619 ts_tablet_manager.cc:1918] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:24:46.179831 20619 log.cc:1196] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/57c0143f868d4207bf76338fe1e9ff59
I20260620 09:24:46.180068 20619 ts_tablet_manager.cc:1939] T 57c0143f868d4207bf76338fe1e9ff59 P 7c3a0cce19dd431f8f5aa0e6624d577b: Deleting consensus metadata
I20260620 09:24:46.180563 20272 catalog_manager.cc:4841] TS 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2:43065): tablet 57c0143f868d4207bf76338fe1e9ff59 (table test-table [id=dba12231562c49779cc4450cf54dc6e5]) successfully deleted
W20260620 09:24:46.248023 20617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:24:46.248210 20617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:24:46.248232 20617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:24:46.249362 20617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:24:46.249399 20617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:24:46.249426 20617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:24:46.250802 20617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:46521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=46801
--tserver_master_addrs=127.4.59.62:35489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:24:46.250978 20617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:24:46.251147 20617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:24:46.255885 20624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:46.255890 20625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:24:46.256271 20627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:24:46.256258 20617 server_base.cc:1034] running on GCE node
I20260620 09:24:46.256489 20617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:24:46.256709 20617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:24:46.257841 20617 hybrid_clock.cc:648] HybridClock initialized: now 1781947486257832 us; error 28 us; skew 500 ppm
Jun 20 09:24:46 dist-test-slave-414w krb5kdc[19707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947486, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:24:46.260316 20617 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:24:46.260658 20617 webserver.cc:460] Webserver started at http://127.4.59.3:46801/ using document root <none> and password file <none>
I20260620 09:24:46.260843 20617 fs_manager.cc:359] Metadata directory not provided
I20260620 09:24:46.260895 20617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:24:46.262159 20617 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:46.262669 20634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:46.262854 20617 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:46.262939 20617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "609c46105ce04dc487af086a83f105a4"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "9d7bd2cb1e7b6c510c7d101d1faacd08"
server_key_iv: "6c9da144fb0198ab9ddac967e74d0a12"
server_key_version: "encryptionkey@0"
I20260620 09:24:46.263218 20617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:24:46.287703 20617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:24:46.289458 20617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:24:46.289563 20617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:24:46.289736 20617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:24:46.289965 20617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:24:46.289994 20617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:46.290014 20617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:24:46.290027 20617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:46.296563 20617 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:46521
I20260620 09:24:46.296629 20747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:46521 every 8 connection(s)
I20260620 09:24:46.296959 20617 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:24:46.300719 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 20617
Jun 20 09:24:46 dist-test-slave-414w krb5kdc[19707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947486, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:24:46.306183 20476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.297728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:49105 (local address 127.4.59.62:35489)
0620 09:24:46.297791 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:46.297792 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:46.297967 (+ 175us) server_negotiation.cc:407] Connection header received
0620 09:24:46.298401 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:46.298402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:46.298412 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:46.298465 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:46.299300 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.299783 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.300747 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.301061 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.301132 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.303471 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:46.303472 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:46.303472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:46.303478 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:46.304812 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:46.305301 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:46.305301 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:46.305302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:46.305371 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:46.305604 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:46.305605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:46.305605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:46.305744 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:46.305804 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:46.306046 (+ 242us) server_negotiation.cc:299] Negotiation successful
0620 09:24:46.306115 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 09:24:46.306381 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.297708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:49105)
0620 09:24:46.297850 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:46.297888 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:46.298323 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:46.298510 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:46.298513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:46.299033 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:24:46.299203 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:46.299207 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.299858 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.299858 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:46.300648 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:46.300650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.301206 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.301208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:46.301340 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.301832 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:24:46.301839 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:24:46.303372 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:24:46.304897 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:46.304904 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:46.304907 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:46.305228 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:46.305445 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:24:46.305446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:24:46.305446 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:24:46.305538 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:24:46.305808 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:24:46.305812 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:24:46.305922 (+ 110us) client_negotiation.cc:769] Sending connection context
0620 09:24:46.305997 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 09:24:46.306155 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260620 09:24:46.306854 20748 heartbeater.cc:347] Connected to a master server at 127.4.59.62:35489
I20260620 09:24:46.306936 20748 heartbeater.cc:464] Registering TS with master...
I20260620 09:24:46.307101 20748 heartbeater.cc:511] Master 127.4.59.62:35489 requested a full tablet report, sending...
I20260620 09:24:46.307487 20285 ts_manager.cc:194] Registered new tserver with Master: 609c46105ce04dc487af086a83f105a4 (127.4.59.3:46521)
I20260620 09:24:46.307984 20285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:49105
I20260620 09:24:46.308354 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.301133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50412 (local address 127.4.59.62:35489)
0620 09:24:46.301389 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:46.301390 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:46.301395 (+ 5us) server_negotiation.cc:407] Connection header received
0620 09:24:46.301410 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:46.301411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:46.301419 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:46.301494 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:46.302055 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.302703 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.303708 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.303998 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.304068 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.305389 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:46.305390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:46.305390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:46.305396 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:46.307183 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:46.307572 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:46.307573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:46.307573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:46.307632 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:46.307826 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:46.307826 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:46.307826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:46.307908 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:46.308021 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:46.308165 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 09:24:46.308244 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":68,"threads_started":1}
I20260620 09:24:46.309058 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:24:46.311308 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.309396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50422 (local address 127.4.59.62:35489)
0620 09:24:46.309429 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:46.309430 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:46.309510 (+ 80us) server_negotiation.cc:407] Connection header received
0620 09:24:46.309571 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:46.309571 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:46.309594 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:46.309648 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:24:46.309730 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.310146 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.310804 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.310984 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.311031 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.311147 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:24:46.311177 (+ 30us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0620 09:24:46.311272 (+ 95us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50422: token expired
Metrics: {"server-negotiator.queue_time_us":17}
W20260620 09:24:46.311394 20752 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50422: token expired
W20260620 09:24:46.311473 20214 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.4.59.62:35489)
I20260620 09:24:46.316717 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.311700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50434 (local address 127.4.59.62:35489)
0620 09:24:46.311729 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:46.311730 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:46.311781 (+ 51us) server_negotiation.cc:407] Connection header received
0620 09:24:46.311825 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:46.311825 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:46.311832 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:46.311879 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:24:46.312291 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.312691 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.313374 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.313538 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.313587 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.314505 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:24:46.314505 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:24:46.314506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:24:46.314511 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:24:46.315841 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:46.316121 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:46.316121 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:46.316121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:46.316165 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:24:46.316300 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:24:46.316300 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:24:46.316300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:24:46.316382 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:24:46.316446 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:46.316529 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 09:24:46.316617 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260620 09:24:46.317742 20285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:24:46.319254 20285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:24:46.321261 20407 tablet_service.cc:1467] Processing CreateTablet for tablet ebccc259c17247fab7e2e6c04a7709da (DEFAULT_TABLE table=test-table [id=15ef3d32777048be95e8056a048d477e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:24:46.321290 20546 tablet_service.cc:1467] Processing CreateTablet for tablet ebccc259c17247fab7e2e6c04a7709da (DEFAULT_TABLE table=test-table [id=15ef3d32777048be95e8056a048d477e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:24:46.321451 20407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebccc259c17247fab7e2e6c04a7709da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:46.321456 20546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebccc259c17247fab7e2e6c04a7709da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:46.322750 20365 tablet_bootstrap.cc:492] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7: Bootstrap starting.
I20260620 09:24:46.322784 20504 tablet_bootstrap.cc:492] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b: Bootstrap starting.
I20260620 09:24:46.323038 20365 tablet_bootstrap.cc:654] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:46.323063 20504 tablet_bootstrap.cc:654] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:46.323518 20504 tablet_bootstrap.cc:492] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b: No bootstrap required, opened a new log
I20260620 09:24:46.323558 20504 ts_tablet_manager.cc:1397] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260620 09:24:46.323912 20504 raft_consensus.cc:348] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:46.324072 20504 raft_consensus.cc:374] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:46.324122 20504 raft_consensus.cc:729] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3a0cce19dd431f8f5aa0e6624d577b, State: Initialized, Role: FOLLOWER
I20260620 09:24:46.324193 20504 consensus_queue.cc:260] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:46.324293 20504 ts_tablet_manager.cc:1428] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 09:24:46.324587 20365 tablet_bootstrap.cc:492] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7: No bootstrap required, opened a new log
I20260620 09:24:46.324627 20365 ts_tablet_manager.cc:1397] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260620 09:24:46.324736 20365 raft_consensus.cc:348] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:46.324790 20365 raft_consensus.cc:374] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:46.324826 20365 raft_consensus.cc:729] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3ee7e86e21548b599c884a61dbb8cf7, State: Initialized, Role: FOLLOWER
I20260620 09:24:46.324882 20365 consensus_queue.cc:260] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:46.324962 20365 ts_tablet_manager.cc:1428] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 09:24:46.326376 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.321028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54684 (local address 127.4.59.3:46521)
0620 09:24:46.323460 (+ 2432us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:46.323464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:46.323473 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:24:46.323504 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:46.323505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:46.323516 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:46.323615 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:46.323751 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.324669 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.325685 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.326136 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.326194 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.326230 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:46.326301 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 09:24:46.326324 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2396,"thread_start_us":74,"threads_started":1}
I20260620 09:24:46.326438 20480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.320943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:46521 (local address 127.0.0.1:54684)
0620 09:24:46.320993 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:46.321003 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:46.321046 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:46.323630 (+ 2584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:46.323630 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:46.323632 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:46.323675 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:46.323676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.324745 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.324746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:46.325614 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:46.325617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.326205 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.326206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:46.326245 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.326248 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:24:46.326290 (+ 42us) client_negotiation.cc:240] Negotiation successful
0620 09:24:46.326372 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 09:24:46.327051 20682 tablet_service.cc:1467] Processing CreateTablet for tablet ebccc259c17247fab7e2e6c04a7709da (DEFAULT_TABLE table=test-table [id=15ef3d32777048be95e8056a048d477e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:24:46.327339 20682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebccc259c17247fab7e2e6c04a7709da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:24:46.330062 20757 tablet_bootstrap.cc:492] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4: Bootstrap starting.
I20260620 09:24:46.330574 20757 tablet_bootstrap.cc:654] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4: Neither blocks nor log segments found. Creating new log.
I20260620 09:24:46.330986 20757 log.cc:824] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4: Log is configured to *not* fsync() on all Append() calls
I20260620 09:24:46.331641 20757 tablet_bootstrap.cc:492] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4: No bootstrap required, opened a new log
I20260620 09:24:46.331698 20757 ts_tablet_manager.cc:1397] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:24:46.333098 20757 raft_consensus.cc:348] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:46.333292 20757 raft_consensus.cc:374] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:24:46.333317 20757 raft_consensus.cc:729] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 609c46105ce04dc487af086a83f105a4, State: Initialized, Role: FOLLOWER
I20260620 09:24:46.333429 20757 consensus_queue.cc:260] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:46.333699 20748 heartbeater.cc:503] Master 127.4.59.62:35489 was elected leader, sending a full tablet report...
I20260620 09:24:46.333704 20757 ts_tablet_manager.cc:1428] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260620 09:24:46.406946 20613 tablet.cc:2369] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 09:24:46.524549 20474 tablet.cc:2369] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 09:24:46.547550 20749 tablet.cc:2369] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:24:46.558192 20759 raft_consensus.cc:482] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:24:46.558332 20759 raft_consensus.cc:504] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:46.558857 20759 leader_election.cc:288] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3ee7e86e21548b599c884a61dbb8cf7 (127.4.59.1:33081), 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2:43065)
I20260620 09:24:46.563023 20618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.559291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:56141 (local address 127.4.59.2:43065)
0620 09:24:46.559358 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:46.559359 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:46.559440 (+ 81us) server_negotiation.cc:407] Connection header received
0620 09:24:46.559468 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:46.559469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:46.559494 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:46.559558 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:46.559697 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.560460 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.562059 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.562654 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.562716 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.562791 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:46.562911 (+ 120us) server_negotiation.cc:299] Negotiation successful
0620 09:24:46.562951 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 09:24:46.563024 20760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.559199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:60349 (local address 127.4.59.1:33081)
0620 09:24:46.559324 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:46.559325 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:46.559337 (+ 12us) server_negotiation.cc:407] Connection header received
0620 09:24:46.559370 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:46.559372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:46.559392 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:46.559493 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:24:46.559694 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.560460 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.562059 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.562620 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.562702 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.562717 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:46.562911 (+ 194us) server_negotiation.cc:299] Negotiation successful
0620 09:24:46.562956 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
I20260620 09:24:46.563035 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.559075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:33081 (local address 127.4.59.3:60349)
0620 09:24:46.559137 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:46.559183 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:46.559284 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:46.559504 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:46.559505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:46.559509 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:46.559599 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:46.559601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.560553 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.560554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:46.561971 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:46.561973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.562728 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.562729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:46.562819 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.562826 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 09:24:46.562888 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 09:24:46.562951 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":151}
I20260620 09:24:46.563042 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.559206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:43065 (local address 127.4.59.3:56141)
0620 09:24:46.559364 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:46.559373 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:24:46.559425 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:24:46.559572 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:24:46.559572 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:24:46.559574 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:24:46.559628 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:46.559629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.560553 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.560554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:46.561971 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:24:46.561973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.562732 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.562733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:24:46.562819 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.562826 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 09:24:46.562888 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 09:24:46.562948 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":48,"thread_start_us":78,"threads_started":1}
I20260620 09:24:46.563457 20566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebccc259c17247fab7e2e6c04a7709da" candidate_uuid: "609c46105ce04dc487af086a83f105a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" is_pre_election: true
I20260620 09:24:46.563472 20427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebccc259c17247fab7e2e6c04a7709da" candidate_uuid: "609c46105ce04dc487af086a83f105a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" is_pre_election: true
I20260620 09:24:46.563585 20566 raft_consensus.cc:2454] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 609c46105ce04dc487af086a83f105a4 in term 0.
I20260620 09:24:46.563623 20427 raft_consensus.cc:2454] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 609c46105ce04dc487af086a83f105a4 in term 0.
I20260620 09:24:46.563827 20638 leader_election.cc:302] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 609c46105ce04dc487af086a83f105a4, 7c3a0cce19dd431f8f5aa0e6624d577b; no voters:
I20260620 09:24:46.563957 20759 raft_consensus.cc:2781] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:24:46.564045 20759 raft_consensus.cc:482] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:24:46.564083 20759 raft_consensus.cc:3037] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:46.564782 20759 raft_consensus.cc:504] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:46.564921 20759 leader_election.cc:288] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [CANDIDATE]: Term 1 election: Requested vote from peers c3ee7e86e21548b599c884a61dbb8cf7 (127.4.59.1:33081), 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2:43065)
I20260620 09:24:46.565109 20427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebccc259c17247fab7e2e6c04a7709da" candidate_uuid: "609c46105ce04dc487af086a83f105a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3ee7e86e21548b599c884a61dbb8cf7"
I20260620 09:24:46.565115 20566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebccc259c17247fab7e2e6c04a7709da" candidate_uuid: "609c46105ce04dc487af086a83f105a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b"
I20260620 09:24:46.565215 20427 raft_consensus.cc:3037] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:46.565232 20566 raft_consensus.cc:3037] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:24:46.566183 20566 raft_consensus.cc:2454] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 609c46105ce04dc487af086a83f105a4 in term 1.
I20260620 09:24:46.566183 20427 raft_consensus.cc:2454] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 609c46105ce04dc487af086a83f105a4 in term 1.
I20260620 09:24:46.566340 20638 leader_election.cc:302] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 609c46105ce04dc487af086a83f105a4, 7c3a0cce19dd431f8f5aa0e6624d577b; no voters:
I20260620 09:24:46.566464 20759 raft_consensus.cc:2781] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:24:46.566610 20759 raft_consensus.cc:686] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [term 1 LEADER]: Becoming Leader. State: Replica: 609c46105ce04dc487af086a83f105a4, State: Running, Role: LEADER
I20260620 09:24:46.566711 20759 consensus_queue.cc:237] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:24:46.567507 20285 catalog_manager.cc:5495] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 609c46105ce04dc487af086a83f105a4 (127.4.59.3). New cstate: current_term: 1 leader_uuid: "609c46105ce04dc487af086a83f105a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } health_report { overall_health: UNKNOWN } } }
I20260620 09:24:46.606535 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.604435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54688 (local address 127.4.59.3:46521)
0620 09:24:46.604496 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 09:24:46.604498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:24:46.604506 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:24:46.604538 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:24:46.604539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:24:46.604556 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:24:46.604614 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:24:46.604731 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.605203 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.605877 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:24:46.606050 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:24:46.606097 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:24:46.606216 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:24:46.606304 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:24:46.606364 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 09:24:46.606389 (+ 25us) server_negotiation.cc:299] Negotiation successful
0620 09:24:46.606404 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 09:24:46.608359 20427 raft_consensus.cc:1261] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Refusing update from remote peer 609c46105ce04dc487af086a83f105a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:24:46.608397 20566 raft_consensus.cc:1261] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b [term 1 FOLLOWER]: Refusing update from remote peer 609c46105ce04dc487af086a83f105a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:24:46.608726 20759 consensus_queue.cc:1035] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:24:46.608858 20762 consensus_queue.cc:1035] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:24:46.611147 20767 mvcc.cc:205] Tried to move back new op lower bound from 7298856905145270272 to 7298856904978235392. Current Snapshot: MvccSnapshot[applied={T|T < 7298856905145270272}]
I20260620 09:24:46.613049 20285 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:24:46.613128 20285 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:24:46.613895 20285 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Sending DeleteTablet for 3 replicas of tablet ebccc259c17247fab7e2e6c04a7709da
I20260620 09:24:46.614183 20682 tablet_service.cc:1514] Processing DeleteTablet for tablet ebccc259c17247fab7e2e6c04a7709da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:46 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:54684
I20260620 09:24:46.614271 20407 tablet_service.cc:1514] Processing DeleteTablet for tablet ebccc259c17247fab7e2e6c04a7709da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:46 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:43042
I20260620 09:24:46.614360 20546 tablet_service.cc:1514] Processing DeleteTablet for tablet ebccc259c17247fab7e2e6c04a7709da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:24:46 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:40544
I20260620 09:24:46.614383 20775 tablet_replica.cc:330] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4: stopping tablet replica
I20260620 09:24:46.614444 20619 tablet_replica.cc:330] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b: stopping tablet replica
I20260620 09:24:46.614470 20775 raft_consensus.cc:2229] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:24:46.614480 20619 raft_consensus.cc:2229] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:24:46.614475 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 20323
I20260620 09:24:46.614529 20619 raft_consensus.cc:2258] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:24:46.614557 20775 raft_consensus.cc:2258] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:24:46.615093 20775 ts_tablet_manager.cc:1905] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:24:46.615440 20619 ts_tablet_manager.cc:1905] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:24:46.616833 20775 ts_tablet_manager.cc:1918] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:24:46.616916 20775 log.cc:1196] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/ebccc259c17247fab7e2e6c04a7709da
I20260620 09:24:46.616910 20619 ts_tablet_manager.cc:1918] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:24:46.616958 20619 log.cc:1196] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/ebccc259c17247fab7e2e6c04a7709da
I20260620 09:24:46.617316 20619 ts_tablet_manager.cc:1939] T ebccc259c17247fab7e2e6c04a7709da P 7c3a0cce19dd431f8f5aa0e6624d577b: Deleting consensus metadata
I20260620 09:24:46.617376 20775 ts_tablet_manager.cc:1939] T ebccc259c17247fab7e2e6c04a7709da P 609c46105ce04dc487af086a83f105a4: Deleting consensus metadata
I20260620 09:24:46.617660 20272 catalog_manager.cc:4841] TS 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2:43065): tablet ebccc259c17247fab7e2e6c04a7709da (table test-table [id=15ef3d32777048be95e8056a048d477e]) successfully deleted
I20260620 09:24:46.617898 20272 catalog_manager.cc:4841] TS 609c46105ce04dc487af086a83f105a4 (127.4.59.3:46521): tablet ebccc259c17247fab7e2e6c04a7709da (table test-table [id=15ef3d32777048be95e8056a048d477e]) successfully deleted
W20260620 09:24:46.620734 20272 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.59.1:33081 (error 108)
I20260620 09:24:46.621181 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 20477
I20260620 09:24:46.621198 20480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:24:46.621078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:33081 (local address 127.0.0.1:43052)
0620 09:24:46.621114 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0620 09:24:46.621169 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:33081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260620 09:24:46.622356 20272 catalog_manager.cc:4584] TS c3ee7e86e21548b599c884a61dbb8cf7 (127.4.59.1:33081): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ebccc259c17247fab7e2e6c04a7709da: Network error: Client connection negotiation failed: client connection to 127.4.59.1:33081: connect: Connection refused (error 111)
I20260620 09:24:46.625574 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 20617
I20260620 09:24:46.630204 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 20252
I20260620 09:25:06.635654 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:35489
--webserver_interface=127.4.59.62
--webserver_port=43619
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:35489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:06.714350 20778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:06.714541 20778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:06.714563 20778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:06.715731 20778 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 09:25:06.715775 20778 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 09:25:06.715788 20778 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 09:25:06.715801 20778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:06.715813 20778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:06.715826 20778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:06.715866 20778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:06.715896 20778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:06.717386 20778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:35489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:35489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=43619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:06.717617 20778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:06.717861 20778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:06.722751 20784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:06.722741 20783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:06.722783 20778 server_base.cc:1034] running on GCE node
W20260620 09:25:06.722733 20786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:06.723196 20778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:06.723505 20778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:06.724668 20778 hybrid_clock.cc:648] HybridClock initialized: now 1781947506724649 us; error 35 us; skew 500 ppm
Jun 20 09:25:06 dist-test-slave-414w krb5kdc[19707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947506, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:06.727190 20778 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:25:06.727632 20778 webserver.cc:460] Webserver started at http://127.4.59.62:43619/ using document root <none> and password file <none>
I20260620 09:25:06.727881 20778 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:06.727928 20778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:06.729249 20778 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:25:06.730015 20793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:06.730196 20778 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:06.730296 20778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "44493f35a1514b2ba46b0e3d05665518"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "faae8e09901bf012f1d3610bc33f4470"
server_key_iv: "625ffa47380b5158e9470664a05d3a85"
server_key_version: "encryptionkey@0"
I20260620 09:25:06.730631 20778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:06.756290 20778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:06.758365 20778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:06.758526 20778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:06.762982 20778 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:35489
I20260620 09:25:06.763027 20845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:35489 every 8 connection(s)
I20260620 09:25:06.763404 20778 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:06.764835 20846 sys_catalog.cc:263] Verifying existing consensus state
I20260620 09:25:06.765543 20846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Bootstrap starting.
I20260620 09:25:06.770455 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 20778
I20260620 09:25:06.770767 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:33081
--local_ip_for_outbound_sockets=127.4.59.1
--tserver_master_addrs=127.4.59.62:35489
--webserver_port=46673
--webserver_interface=127.4.59.1
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:25:06.774910 20846 log.cc:824] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:06.778064 20846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 09:25:06.778447 20846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Bootstrap complete.
I20260620 09:25:06.781942 20846 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } }
I20260620 09:25:06.782316 20846 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44493f35a1514b2ba46b0e3d05665518, State: Initialized, Role: FOLLOWER
I20260620 09:25:06.782501 20846 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } }
I20260620 09:25:06.782617 20846 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:06.782696 20846 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:06.782753 20846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 2 FOLLOWER]: Advancing to term 3
I20260620 09:25:06.784101 20846 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } }
I20260620 09:25:06.784214 20846 leader_election.cc:302] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44493f35a1514b2ba46b0e3d05665518; no voters:
I20260620 09:25:06.784366 20846 leader_election.cc:288] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 09:25:06.784472 20850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 09:25:06.784646 20850 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [term 3 LEADER]: Becoming Leader. State: Replica: 44493f35a1514b2ba46b0e3d05665518, State: Running, Role: LEADER
I20260620 09:25:06.784708 20846 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:06.784783 20850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } }
I20260620 09:25:06.785269 20850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44493f35a1514b2ba46b0e3d05665518. Latest consensus state: current_term: 3 leader_uuid: "44493f35a1514b2ba46b0e3d05665518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } } }
I20260620 09:25:06.785338 20850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:06.785749 20851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "44493f35a1514b2ba46b0e3d05665518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44493f35a1514b2ba46b0e3d05665518" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35489 } } }
I20260620 09:25:06.785810 20851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:06.785943 20855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:06.787084 20855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:06.787179 20855 catalog_manager.cc:1259] Loaded cluster ID: a588143cad7c4a15bdbabdb94789f318
I20260620 09:25:06.787204 20855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:06.788751 20855 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:06.788923 20855 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Loaded TSK: 0
I20260620 09:25:06.788966 20855 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 44493f35a1514b2ba46b0e3d05665518: Loaded TSK: 1
F20260620 09:25:06.789363 20855 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f16623c49ad google::LogMessage::Fail() at ??:0
@ 0x7f16623c788a google::LogMessage::SendToLog() at ??:0
@ 0x7f16623c44de google::LogMessage::Flush() at ??:0
@ 0x7f16623c5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1664b31104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f1664b2ed41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f166a31e1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f166a31e48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f166a31e594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f166a2e4c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f166a2eead0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f16642920d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1664287954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f16650eb6db start_thread at ??:0
@ 0x7f1660ec261f clone at ??:0
W20260620 09:25:06.896530 20848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:06.896800 20848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:06.896837 20848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:06.898624 20848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:06.898685 20848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:06.898725 20848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:25:06.900859 20848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:33081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=46673
--tserver_master_addrs=127.4.59.62:35489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:06.901100 20848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:06.901381 20848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:06.906497 20887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:06.906522 20890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:06.906651 20848 server_base.cc:1034] running on GCE node
W20260620 09:25:06.906711 20888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:06.906955 20848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:06.907202 20848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:06.908370 20848 hybrid_clock.cc:648] HybridClock initialized: now 1781947506908356 us; error 31 us; skew 500 ppm
Jun 20 09:25:06 dist-test-slave-414w krb5kdc[19707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947506, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:06.911104 20848 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:25:06.911636 20848 webserver.cc:460] Webserver started at http://127.4.59.1:46673/ using document root <none> and password file <none>
I20260620 09:25:06.911882 20848 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:06.911949 20848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:06.913763 20848 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:25:06.914502 20897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:06.914707 20848 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:06.914805 20848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "c3ee7e86e21548b599c884a61dbb8cf7"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "08d28a5b47c6bc799f88e924e0e01c07"
server_key_iv: "5756359c24b981f200a195d032719fed"
server_key_version: "encryptionkey@0"
I20260620 09:25:06.915248 20848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:06.954913 20848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:06.957451 20848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:06.957607 20848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:06.957863 20848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:06.958321 20904 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 09:25:06.960517 20848 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 09:25:06.960602 20848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 09:25:06.960644 20848 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 09:25:06.961367 20848 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 09:25:06.961422 20848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:06.961496 20904 tablet_bootstrap.cc:492] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7: Bootstrap starting.
I20260620 09:25:06.968382 20848 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:33081
I20260620 09:25:06.968523 21011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:33081 every 8 connection(s)
I20260620 09:25:06.968803 20848 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:25:06.972533 20904 log.cc:824] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:06.975172 20904 tablet_bootstrap.cc:492] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 09:25:06.975481 20904 tablet_bootstrap.cc:492] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7: Bootstrap complete.
I20260620 09:25:06.976747 20904 ts_tablet_manager.cc:1397] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260620 09:25:06.977054 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 20848
I20260620 09:25:06.977355 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:43065
--local_ip_for_outbound_sockets=127.4.59.2
--tserver_master_addrs=127.4.59.62:35489
--webserver_port=40709
--webserver_interface=127.4.59.2
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:25:06.978508 20904 raft_consensus.cc:348] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:25:06.978931 20904 raft_consensus.cc:729] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3ee7e86e21548b599c884a61dbb8cf7, State: Initialized, Role: FOLLOWER
I20260620 09:25:06.979080 20904 consensus_queue.cc:260] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:25:06.979337 20904 ts_tablet_manager.cc:1428] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
W20260620 09:25:07.021138 21014 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:06.969537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:42819)
0620 09:25:06.969737 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:06.969773 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:06.970209 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:07.020953 (+ 50744us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":119,"threads_started":1}
W20260620 09:25:07.021433 21012 heartbeater.cc:644] Failed to heartbeat to 127.4.59.62:35489 (0 consecutive failures): Network error: Failed to ping master at 127.4.59.62:35489: Client connection negotiation failed: client connection to 127.4.59.62:35489: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 09:25:07.021947 21014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:07.021658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:45955)
0620 09:25:07.021783 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:07.021899 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260620 09:25:07.022578 21014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:07.022417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:36825)
0620 09:25:07.022503 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:07.022538 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260620 09:25:07.022715 21012 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 09:25:07.060618 21016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:07.060853 21016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:07.060895 21016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:07.062091 21016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:07.062155 21016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:07.062199 21016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:25:07.063611 21016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:43065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=40709
--tserver_master_addrs=127.4.59.62:35489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:07.063864 21016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:07.064085 21016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 09:25:07.068878 21016 server_base.cc:1034] running on GCE node
W20260620 09:25:07.068853 21025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:07.068902 21023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:07.068856 21022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:07.069283 21016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:07.069473 21016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:07.070606 21016 hybrid_clock.cc:648] HybridClock initialized: now 1781947507070594 us; error 28 us; skew 500 ppm
Jun 20 09:25:07 dist-test-slave-414w krb5kdc[19707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947507, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:07.072957 21016 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:25:07.073402 21016 webserver.cc:460] Webserver started at http://127.4.59.2:40709/ using document root <none> and password file <none>
I20260620 09:25:07.073634 21016 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:07.073678 21016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:07.074954 21016 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:07.075490 21032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:07.075668 21016 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:07.075765 21016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "7c3a0cce19dd431f8f5aa0e6624d577b"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "01495d9d13a8f984c9f2e6c77e889514"
server_key_iv: "66ce2fdf78da01c3de8578b465ca096b"
server_key_version: "encryptionkey@0"
I20260620 09:25:07.076071 21016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:07.111590 21016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:07.113564 21016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:07.113720 21016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:07.113947 21016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:07.114230 21016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:07.114284 21016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:07.114324 21016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:07.114353 21016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:07.120157 21016 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:43065
I20260620 09:25:07.120204 21145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:43065 every 8 connection(s)
I20260620 09:25:07.120580 21016 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:25:07.121546 21148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:07.121224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:49905)
0620 09:25:07.121358 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:07.121418 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
W20260620 09:25:07.121925 21146 heartbeater.cc:644] Failed to heartbeat to 127.4.59.62:35489 (0 consecutive failures): Network error: Failed to ping master at 127.4.59.62:35489: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
I20260620 09:25:07.122182 21148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:07.122116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:39363)
0620 09:25:07.122142 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:07.122166 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260620 09:25:07.122429 21148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:07.122377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:49521)
0620 09:25:07.122399 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:07.122417 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260620 09:25:07.122537 21146 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 09:25:07.122881 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 21016
I20260620 09:25:07.123139 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:46521
--local_ip_for_outbound_sockets=127.4.59.3
--tserver_master_addrs=127.4.59.62:35489
--webserver_port=46801
--webserver_interface=127.4.59.3
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:07.203222 21149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:07.203449 21149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:07.203485 21149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:07.204636 21149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:07.204701 21149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:07.204746 21149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:25:07.206184 21149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:46521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=46801
--tserver_master_addrs=127.4.59.62:35489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:07.206435 21149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:07.206662 21149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:07.211588 21155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:07.211678 21154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:07.211768 21149 server_base.cc:1034] running on GCE node
W20260620 09:25:07.211778 21157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:07.212055 21149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:07.212297 21149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:07.213460 21149 hybrid_clock.cc:648] HybridClock initialized: now 1781947507213437 us; error 33 us; skew 500 ppm
Jun 20 09:25:07 dist-test-slave-414w krb5kdc[19707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947507, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:07.215952 21149 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:25:07.216437 21149 webserver.cc:460] Webserver started at http://127.4.59.3:46801/ using document root <none> and password file <none>
I20260620 09:25:07.216656 21149 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:07.216723 21149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:07.218057 21149 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:07.218636 21164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:07.218845 21149 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:07.218952 21149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "609c46105ce04dc487af086a83f105a4"
format_stamp: "Formatted at 2026-06-20 09:24:25 on dist-test-slave-414w"
server_key: "9d7bd2cb1e7b6c510c7d101d1faacd08"
server_key_iv: "6c9da144fb0198ab9ddac967e74d0a12"
server_key_version: "encryptionkey@0"
I20260620 09:25:07.219331 21149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:07.227943 21149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:07.230175 21149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:07.230338 21149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:07.230552 21149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:07.230834 21149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:07.230885 21149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:07.230924 21149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:07.230952 21149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:07.236689 21149 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:46521
I20260620 09:25:07.236723 21277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:46521 every 8 connection(s)
I20260620 09:25:07.237057 21149 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:25:07.237717 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 21149
I20260620 09:25:07.238049 21280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:07.237688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:40257)
0620 09:25:07.237838 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:07.237910 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
W20260620 09:25:07.238408 21278 heartbeater.cc:644] Failed to heartbeat to 127.4.59.62:35489 (0 consecutive failures): Network error: Failed to ping master at 127.4.59.62:35489: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
I20260620 09:25:07.238667 21280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:07.238575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:42737)
0620 09:25:07.238621 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:07.238650 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 09:25:07.238930 21280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:07.238845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:58125)
0620 09:25:07.238885 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:07.238907 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260620 09:25:07.239023 21278 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 09:25:08.023409 21282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:08.023167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:59673)
0620 09:25:08.023295 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:08.023346 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260620 09:25:08.123036 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:08.122813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:37281)
0620 09:25:08.122918 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:08.122977 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":55,"threads_started":1}
I20260620 09:25:08.239495 21284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:08.239310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:56293)
0620 09:25:08.239421 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:08.239453 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":53,"threads_started":1}
I20260620 09:25:08.522289 21285 raft_consensus.cc:482] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:25:08.522452 21285 raft_consensus.cc:504] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:25:08.522810 21285 leader_election.cc:288] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 609c46105ce04dc487af086a83f105a4 (127.4.59.3:46521), 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2:43065)
Jun 20 09:25:08 dist-test-slave-414w krb5kdc[19707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947506, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.2@KRBTEST.COM
Jun 20 09:25:08 dist-test-slave-414w krb5kdc[19707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947506, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.3@KRBTEST.COM
I20260620 09:25:08.533144 21287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:08.523207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:43065 (local address 127.4.59.1:50651)
0620 09:25:08.523310 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:08.523324 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:08.523388 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:08.524104 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:08.524105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:08.524414 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:08.524606 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:08.524611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:08.525756 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:08.525757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:08.526540 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:08.526543 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:08.527280 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:08.527281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:08.527346 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:08.527864 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:08.527872 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:08.529427 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:08.531429 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:08.531436 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:08.531438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:08.531734 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:08.531968 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:08.531969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:08.531969 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:08.532103 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:08.532593 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:08.532600 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:08.532748 (+ 148us) client_negotiation.cc:769] Sending connection context
0620 09:25:08.532866 (+ 118us) client_negotiation.cc:240] Negotiation successful
0620 09:25:08.533036 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":57,"thread_start_us":55,"threads_started":1}
I20260620 09:25:08.533272 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:08.523300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:50651 (local address 127.4.59.2:43065)
0620 09:25:08.523437 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:08.523447 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:08.523474 (+ 27us) server_negotiation.cc:407] Connection header received
0620 09:25:08.523889 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:08.523891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:08.523949 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:08.524106 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:08.524727 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:08.525650 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:08.526633 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:08.527076 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:08.527304 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:08.529529 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:08.529533 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:08.529533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:08.529550 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:08.531326 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:08.531817 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:08.531820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:08.531820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:08.531900 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:08.532179 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:08.532180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:08.532180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:08.532502 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:08.532590 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:08.532989 (+ 399us) server_negotiation.cc:299] Negotiation successful
0620 09:25:08.533133 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":47,"threads_started":1}
I20260620 09:25:08.533494 21286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:08.523138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:53251 (local address 127.4.59.3:46521)
0620 09:25:08.523326 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:08.523345 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:08.523373 (+ 28us) server_negotiation.cc:407] Connection header received
0620 09:25:08.523811 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:08.523813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:08.523853 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:08.523962 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:08.524726 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:08.525653 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:08.526639 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:08.527026 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:08.527153 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:08.529961 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:08.529964 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:08.529965 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:08.529976 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:08.531967 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:08.532319 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:08.532322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:08.532322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:08.532401 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:08.532606 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:08.532607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:08.532607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:08.532901 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:08.532976 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:08.533249 (+ 273us) server_negotiation.cc:299] Negotiation successful
0620 09:25:08.533399 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260620 09:25:08.533375 21282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:08.523020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:46521 (local address 127.4.59.1:53251)
0620 09:25:08.523103 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:08.523126 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:08.523182 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:08.524027 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:08.524030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:08.524413 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:08.524606 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:08.524612 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:08.525760 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:08.525761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:08.526541 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:08.526542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:08.527132 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:08.527134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:08.527250 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:08.527919 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:08.527924 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:08.529819 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:08.532037 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:08.532038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:08.532038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:08.532239 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:08.532470 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:08.532470 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:08.532471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:08.532541 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:08.532990 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:08.532991 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:08.533050 (+ 59us) client_negotiation.cc:769] Sending connection context
0620 09:25:08.533162 (+ 112us) client_negotiation.cc:240] Negotiation successful
0620 09:25:08.533258 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":6}
W20260620 09:25:08.534756 20901 leader_election.cc:341] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2:43065): Not found: Tablet not found: ebccc259c17247fab7e2e6c04a7709da
W20260620 09:25:08.534824 20901 leader_election.cc:341] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 609c46105ce04dc487af086a83f105a4 (127.4.59.3:46521): Not found: Tablet not found: ebccc259c17247fab7e2e6c04a7709da
I20260620 09:25:08.534842 20901 leader_election.cc:302] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3ee7e86e21548b599c884a61dbb8cf7; no voters: 609c46105ce04dc487af086a83f105a4, 7c3a0cce19dd431f8f5aa0e6624d577b
I20260620 09:25:08.534937 21285 raft_consensus.cc:2726] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 09:25:09.023999 21282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:09.023873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:59467)
0620 09:25:09.023922 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:09.023962 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 09:25:09.123754 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:09.123540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:52727)
0620 09:25:09.123673 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:09.123708 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260620 09:25:09.240209 21290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:09.239973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:55681)
0620 09:25:09.240128 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:09.240161 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260620 09:25:10.024691 21291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:10.024490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:39925)
0620 09:25:10.024608 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:10.024646 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260620 09:25:10.079107 21292 raft_consensus.cc:482] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:25:10.079200 21292 raft_consensus.cc:504] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:25:10.079489 21292 leader_election.cc:288] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 609c46105ce04dc487af086a83f105a4 (127.4.59.3:46521), 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2:43065)
W20260620 09:25:10.079862 20901 leader_election.cc:341] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 609c46105ce04dc487af086a83f105a4 (127.4.59.3:46521): Not found: Tablet not found: ebccc259c17247fab7e2e6c04a7709da
W20260620 09:25:10.079942 20901 leader_election.cc:341] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2:43065): Not found: Tablet not found: ebccc259c17247fab7e2e6c04a7709da
I20260620 09:25:10.079959 20901 leader_election.cc:302] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3ee7e86e21548b599c884a61dbb8cf7; no voters: 609c46105ce04dc487af086a83f105a4, 7c3a0cce19dd431f8f5aa0e6624d577b
I20260620 09:25:10.080044 21292 raft_consensus.cc:2726] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 09:25:10.124387 21293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:10.124170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:39351)
0620 09:25:10.124303 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:10.124340 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":59,"threads_started":1}
I20260620 09:25:10.240932 21294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:10.240706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:49729)
0620 09:25:10.240859 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:10.240893 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":72,"threads_started":1}
I20260620 09:25:11.025452 21295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:11.025230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:55667)
0620 09:25:11.025371 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:11.025406 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260620 09:25:11.125223 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:11.124929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:46381)
0620 09:25:11.125078 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:11.125117 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260620 09:25:11.241618 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:11.241376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:59287)
0620 09:25:11.241528 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:11.241583 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
I20260620 09:25:12.026238 21298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:12.026007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:34699)
0620 09:25:12.026142 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:12.026178 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":59,"threads_started":1}
I20260620 09:25:12.125876 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:12.125672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:50641)
0620 09:25:12.125800 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:12.125838 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260620 09:25:12.242255 21300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:12.242042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:49045)
0620 09:25:12.242175 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:12.242209 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
I20260620 09:25:13.026925 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:13.026702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:39413)
0620 09:25:13.026843 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:13.026876 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260620 09:25:13.076872 21302 raft_consensus.cc:482] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:25:13.077006 21302 raft_consensus.cc:504] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:25:13.077219 21302 leader_election.cc:288] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 609c46105ce04dc487af086a83f105a4 (127.4.59.3:46521), 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2:43065)
W20260620 09:25:13.077625 20901 leader_election.cc:341] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2:43065): Not found: Tablet not found: ebccc259c17247fab7e2e6c04a7709da
W20260620 09:25:13.077678 20901 leader_election.cc:341] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 609c46105ce04dc487af086a83f105a4 (127.4.59.3:46521): Not found: Tablet not found: ebccc259c17247fab7e2e6c04a7709da
I20260620 09:25:13.077692 20901 leader_election.cc:302] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3ee7e86e21548b599c884a61dbb8cf7; no voters: 609c46105ce04dc487af086a83f105a4, 7c3a0cce19dd431f8f5aa0e6624d577b
I20260620 09:25:13.077755 21302 raft_consensus.cc:2726] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 09:25:13.126530 21303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:13.126317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:35533)
0620 09:25:13.126457 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:13.126493 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260620 09:25:13.242944 21304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:13.242737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:50467)
0620 09:25:13.242861 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:13.242900 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260620 09:25:14.027622 21305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:14.027399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:54359)
0620 09:25:14.027538 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:14.027577 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":57,"threads_started":1}
I20260620 09:25:14.127313 21306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:14.127048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:55825)
0620 09:25:14.127205 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:14.127249 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":75,"threads_started":1}
I20260620 09:25:14.243693 21307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:14.243419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:49005)
0620 09:25:14.243600 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:14.243638 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260620 09:25:15.028295 21308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:15.028078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:57757)
0620 09:25:15.028214 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:15.028250 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260620 09:25:15.128073 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:15.127834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:41089)
0620 09:25:15.127983 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:15.128030 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
I20260620 09:25:15.244310 21310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:15.244114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:40059)
0620 09:25:15.244240 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:15.244272 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260620 09:25:16.028962 21311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:16.028753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:44205)
0620 09:25:16.028881 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:16.028917 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":59,"threads_started":1}
I20260620 09:25:16.128794 21312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:16.128540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:60731)
0620 09:25:16.128679 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:16.128749 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":59,"threads_started":1}
I20260620 09:25:16.244963 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:16.244740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:54225)
0620 09:25:16.244887 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:16.244925 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260620 09:25:17.029737 21314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:17.029500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:47753)
0620 09:25:17.029643 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:17.029680 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":86,"threads_started":1}
I20260620 09:25:17.129616 21315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:17.129341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:41167)
0620 09:25:17.129485 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:17.129521 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":59,"threads_started":1}
I20260620 09:25:17.244210 21316 raft_consensus.cc:482] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:25:17.244297 21316 raft_consensus.cc:504] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:25:17.244495 21316 leader_election.cc:288] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 609c46105ce04dc487af086a83f105a4 (127.4.59.3:46521), 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2:43065)
W20260620 09:25:17.245100 20901 leader_election.cc:341] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 609c46105ce04dc487af086a83f105a4 (127.4.59.3:46521): Not found: Tablet not found: ebccc259c17247fab7e2e6c04a7709da
W20260620 09:25:17.245208 20901 leader_election.cc:341] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2:43065): Not found: Tablet not found: ebccc259c17247fab7e2e6c04a7709da
I20260620 09:25:17.245245 20901 leader_election.cc:302] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3ee7e86e21548b599c884a61dbb8cf7; no voters: 609c46105ce04dc487af086a83f105a4, 7c3a0cce19dd431f8f5aa0e6624d577b
I20260620 09:25:17.245314 21316 raft_consensus.cc:2726] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 09:25:17.245540 21317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:17.245348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:39603)
0620 09:25:17.245459 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:17.245495 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":68,"threads_started":1}
I20260620 09:25:18.030440 21318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:18.030233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:51971)
0620 09:25:18.030358 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:18.030394 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":58,"threads_started":1}
I20260620 09:25:18.130316 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:18.130098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:60411)
0620 09:25:18.130242 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:18.130277 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260620 09:25:18.246114 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:18.245895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:56521)
0620 09:25:18.246034 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:18.246069 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
I20260620 09:25:19.031095 21321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:19.030889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:32925)
0620 09:25:19.031017 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:19.031050 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":75,"threads_started":1}
I20260620 09:25:19.131036 21322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:19.130791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:33429)
0620 09:25:19.130928 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:19.130967 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":61,"threads_started":1}
I20260620 09:25:19.246873 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:19.246633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:53609)
0620 09:25:19.246787 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:19.246827 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
I20260620 09:25:20.031800 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:20.031572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:39511)
0620 09:25:20.031712 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:20.031753 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":57,"threads_started":1}
I20260620 09:25:20.131686 21325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:20.131483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:49189)
0620 09:25:20.131610 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:20.131645 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":59,"threads_started":1}
I20260620 09:25:20.247648 21326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:20.247416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:44899)
0620 09:25:20.247566 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:20.247607 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260620 09:25:21.032582 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:21.032318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:41935)
0620 09:25:21.032488 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:21.032534 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":65,"threads_started":1}
I20260620 09:25:21.132450 21328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:21.132193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:43919)
0620 09:25:21.132349 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:21.132406 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
I20260620 09:25:21.248406 21329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:21.248166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:53957)
0620 09:25:21.248317 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:21.248361 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":76,"threads_started":1}
I20260620 09:25:21.497988 21330 raft_consensus.cc:482] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:25:21.498127 21330 raft_consensus.cc:504] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609c46105ce04dc487af086a83f105a4" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 46521 } } peers { permanent_uuid: "c3ee7e86e21548b599c884a61dbb8cf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 33081 } } peers { permanent_uuid: "7c3a0cce19dd431f8f5aa0e6624d577b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 43065 } }
I20260620 09:25:21.498364 21330 leader_election.cc:288] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 609c46105ce04dc487af086a83f105a4 (127.4.59.3:46521), 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2:43065)
W20260620 09:25:21.498873 20901 leader_election.cc:341] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c3a0cce19dd431f8f5aa0e6624d577b (127.4.59.2:43065): Not found: Tablet not found: ebccc259c17247fab7e2e6c04a7709da
W20260620 09:25:21.498946 20901 leader_election.cc:341] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 609c46105ce04dc487af086a83f105a4 (127.4.59.3:46521): Not found: Tablet not found: ebccc259c17247fab7e2e6c04a7709da
I20260620 09:25:21.498965 20901 leader_election.cc:302] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3ee7e86e21548b599c884a61dbb8cf7; no voters: 609c46105ce04dc487af086a83f105a4, 7c3a0cce19dd431f8f5aa0e6624d577b
I20260620 09:25:21.499074 21330 raft_consensus.cc:2726] T ebccc259c17247fab7e2e6c04a7709da P c3ee7e86e21548b599c884a61dbb8cf7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 09:25:22.033552 21331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:22.033314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.1:48579)
0620 09:25:22.033460 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:22.033507 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260620 09:25:22.133448 21332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:22.133074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.2:41163)
0620 09:25:22.133358 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:22.133401 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":97,"threads_started":1}
I20260620 09:25:22.239892 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 20848
I20260620 09:25:22.245075 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 21016
I20260620 09:25:22.249257 21333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:22.248985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35489 (local address 127.4.59.3:51463)
0620 09:25:22.249119 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:22.249208 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.62:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260620 09:25:22.249864 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 21149
2026-06-20T09:25:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58401 ms)
[----------] 30 tests from SecurityITest (260191 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:22 dist-test-slave-414w krb5kdc[21336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:22 dist-test-slave-414w krb5kdc[21336](info): set up 2 sockets
Jun 20 09:25:22 dist-test-slave-414w krb5kdc[21336](info): commencing operation
krb5kdc: starting...
W20260620 09:25:23.285123 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:23 dist-test-slave-414w krb5kdc[21336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:25:23.397431 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:40941
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:40941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:23.477934 21349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:23.478159 21349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:23.478194 21349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:23.479306 21349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:23.479370 21349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:23.479395 21349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:23.479423 21349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:23.479455 21349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:23.480858 21349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:40941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:40941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:23.481113 21349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:23.481354 21349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:23.486455 21355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:23.486461 21357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:23.486465 21354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:23.486480 21349 server_base.cc:1034] running on GCE node
I20260620 09:25:23.486914 21349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:23.487218 21349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:23.488369 21349 hybrid_clock.cc:648] HybridClock initialized: now 1781947523488348 us; error 39 us; skew 500 ppm
Jun 20 09:25:23 dist-test-slave-414w krb5kdc[21336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947523, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:23.490733 21349 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:25:23.491154 21349 webserver.cc:460] Webserver started at http://127.4.59.62:43305/ using document root <none> and password file <none>
I20260620 09:25:23.491349 21349 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:23.491389 21349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:23.491477 21349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:23.492400 21349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "21ec2802e5be4831985f32b8ac95dcac"
format_stamp: "Formatted at 2026-06-20 09:25:23 on dist-test-slave-414w"
server_key: "e6f45d89622e16cc9b6997e79bf15a88"
server_key_iv: "fbf89d61b7b39845c1aabc9feae1de32"
server_key_version: "encryptionkey@0"
I20260620 09:25:23.492677 21349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "21ec2802e5be4831985f32b8ac95dcac"
format_stamp: "Formatted at 2026-06-20 09:25:23 on dist-test-slave-414w"
server_key: "e6f45d89622e16cc9b6997e79bf15a88"
server_key_iv: "fbf89d61b7b39845c1aabc9feae1de32"
server_key_version: "encryptionkey@0"
I20260620 09:25:23.493955 21349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 09:25:23.494679 21364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:23.494859 21349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 09:25:23.494930 21349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "21ec2802e5be4831985f32b8ac95dcac"
format_stamp: "Formatted at 2026-06-20 09:25:23 on dist-test-slave-414w"
server_key: "e6f45d89622e16cc9b6997e79bf15a88"
server_key_iv: "fbf89d61b7b39845c1aabc9feae1de32"
server_key_version: "encryptionkey@0"
I20260620 09:25:23.495018 21349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:23.508834 21349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:23.510569 21349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:23.510703 21349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:23.514678 21349 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:40941
I20260620 09:25:23.514729 21416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:40941 every 8 connection(s)
I20260620 09:25:23.515029 21349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:23.515841 21417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:23.518041 21417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac: Bootstrap starting.
I20260620 09:25:23.518610 21417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:23.518908 21417 log.cc:824] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:23.519599 21417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac: No bootstrap required, opened a new log
I20260620 09:25:23.521219 21417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21ec2802e5be4831985f32b8ac95dcac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40941 } }
I20260620 09:25:23.521390 21417 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:23.521400 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 21349
I20260620 09:25:23.521436 21417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21ec2802e5be4831985f32b8ac95dcac, State: Initialized, Role: FOLLOWER
I20260620 09:25:23.521520 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:23.521601 21417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21ec2802e5be4831985f32b8ac95dcac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40941 } }
I20260620 09:25:23.521718 21417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:23.521742 4332 external_mini_cluster.cc:1428] Setting key ccde77a348043ce6b143bdcdb1db70a2
I20260620 09:25:23.521763 21417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:23.521816 21417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:23.522883 21417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21ec2802e5be4831985f32b8ac95dcac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40941 } }
I20260620 09:25:23.523000 21417 leader_election.cc:302] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21ec2802e5be4831985f32b8ac95dcac; no voters:
I20260620 09:25:23.523167 21417 leader_election.cc:288] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:23.523332 21422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:23.523417 21417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:23.523494 21422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [term 1 LEADER]: Becoming Leader. State: Replica: 21ec2802e5be4831985f32b8ac95dcac, State: Running, Role: LEADER
I20260620 09:25:23.523622 21422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21ec2802e5be4831985f32b8ac95dcac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40941 } }
I20260620 09:25:23.523998 21424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21ec2802e5be4831985f32b8ac95dcac. Latest consensus state: current_term: 1 leader_uuid: "21ec2802e5be4831985f32b8ac95dcac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21ec2802e5be4831985f32b8ac95dcac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40941 } } }
I20260620 09:25:23.524151 21424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:23.524554 21423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21ec2802e5be4831985f32b8ac95dcac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21ec2802e5be4831985f32b8ac95dcac" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40941 } } }
I20260620 09:25:23.524590 21431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:23.524713 21423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:23.525200 21431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:23.526871 21431 catalog_manager.cc:1347] Generated new cluster ID: 84941ea0feb64059a31d0136190c8b4b
I20260620 09:25:23.526924 21431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 09:25:23 dist-test-slave-414w krb5kdc[21336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:23.532985 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:23.522208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37538 (local address 127.4.59.62:40941)
0620 09:25:23.522427 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:23.522435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:23.522462 (+ 27us) server_negotiation.cc:407] Connection header received
0620 09:25:23.522549 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:23.522554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:23.522593 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:23.522698 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:23.523515 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:23.525043 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:23.526097 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:23.526634 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:23.526756 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:23.529314 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:23.529318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:23.529318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:23.529329 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:23.531212 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:23.531557 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:23.531560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:23.531561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:23.531671 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:23.531836 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:23.531836 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:23.531836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:23.532090 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:23.532219 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:23.532659 (+ 440us) server_negotiation.cc:299] Negotiation successful
0620 09:25:23.532782 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":138,"threads_started":1}
I20260620 09:25:23.550308 21431 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:23.550825 21431 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:23.554041 21431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac: Generated new TSK 0
I20260620 09:25:23.554237 21431 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:25:23.581094 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:40941
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:23.672647 21443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:23.672851 21443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:23.672875 21443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:23.674129 21443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:23.674182 21443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:23.674211 21443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:25:23.675639 21443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:40941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:23.675873 21443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:23.676129 21443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:23.681772 21451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:23.681803 21448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:23.681782 21449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:23.682456 21443 server_base.cc:1034] running on GCE node
I20260620 09:25:23.682675 21443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:23.682916 21443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:23.684063 21443 hybrid_clock.cc:648] HybridClock initialized: now 1781947523684046 us; error 33 us; skew 500 ppm
Jun 20 09:25:23 dist-test-slave-414w krb5kdc[21336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947523, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:23.686748 21443 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:25:23.687160 21443 webserver.cc:460] Webserver started at http://127.4.59.1:45919/ using document root <none> and password file <none>
I20260620 09:25:23.687404 21443 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:23.687470 21443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:23.687615 21443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:23.688755 21443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "0d01242fdcce44e7a54d9d1b8b2dfafb"
format_stamp: "Formatted at 2026-06-20 09:25:23 on dist-test-slave-414w"
server_key: "9425fc6917900269130b6290381ad9ca"
server_key_iv: "b567e537fa80b5baf1180f5963167286"
server_key_version: "encryptionkey@0"
I20260620 09:25:23.689132 21443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "0d01242fdcce44e7a54d9d1b8b2dfafb"
format_stamp: "Formatted at 2026-06-20 09:25:23 on dist-test-slave-414w"
server_key: "9425fc6917900269130b6290381ad9ca"
server_key_iv: "b567e537fa80b5baf1180f5963167286"
server_key_version: "encryptionkey@0"
I20260620 09:25:23.690547 21443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:23.691540 21458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:23.691730 21443 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:23.691824 21443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "0d01242fdcce44e7a54d9d1b8b2dfafb"
format_stamp: "Formatted at 2026-06-20 09:25:23 on dist-test-slave-414w"
server_key: "9425fc6917900269130b6290381ad9ca"
server_key_iv: "b567e537fa80b5baf1180f5963167286"
server_key_version: "encryptionkey@0"
I20260620 09:25:23.691907 21443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:23.713879 21443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:23.715758 21443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:23.715926 21443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:23.716163 21443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:23.716481 21443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:23.716536 21443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:23.716576 21443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:23.716606 21443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:23.722396 21443 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:36153
I20260620 09:25:23.722445 21571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:36153 every 8 connection(s)
I20260620 09:25:23.722756 21443 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:25:23.726141 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 21443
I20260620 09:25:23.726240 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:25:23.726442 4332 external_mini_cluster.cc:1428] Setting key be0fd6433dba2843392148ba1230f3e0
Jun 20 09:25:23 dist-test-slave-414w krb5kdc[21336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947523, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:23.732954 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:23.723519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:45411 (local address 127.4.59.62:40941)
0620 09:25:23.723588 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:23.723590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:23.723816 (+ 226us) server_negotiation.cc:407] Connection header received
0620 09:25:23.724356 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:23.724357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:23.724370 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:23.724421 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:23.725375 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:23.725889 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:23.727229 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:23.727447 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:23.727536 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:23.729819 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:23.729821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:23.729822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:23.729830 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:23.731572 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:23.731959 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:23.731960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:23.731961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:23.732036 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:23.732283 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:23.732285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:23.732285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:23.732438 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:23.732512 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:23.732753 (+ 241us) server_negotiation.cc:299] Negotiation successful
0620 09:25:23.732853 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 09:25:23.732983 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:23.723522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40941 (local address 127.4.59.1:45411)
0620 09:25:23.723675 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:23.723709 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:23.724270 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:23.724473 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:23.724476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:23.725086 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:23.725293 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:23.725298 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:23.726001 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:23.726002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:23.727122 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:23.727125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:23.727553 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:23.727554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:23.727690 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:23.728303 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:23.728314 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:23.729727 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:23.731662 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:23.731667 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:23.731669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:23.731885 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:23.732106 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:23.732107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:23.732107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:23.732205 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:23.732503 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:23.732508 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:23.732618 (+ 110us) client_negotiation.cc:769] Sending connection context
0620 09:25:23.732701 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 09:25:23.732823 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260620 09:25:23.733623 21572 heartbeater.cc:347] Connected to a master server at 127.4.59.62:40941
I20260620 09:25:23.733705 21572 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:23.733929 21572 heartbeater.cc:511] Master 127.4.59.62:40941 requested a full tablet report, sending...
I20260620 09:25:23.734685 21381 ts_manager.cc:194] Registered new tserver with Master: 0d01242fdcce44e7a54d9d1b8b2dfafb (127.4.59.1:36153)
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
I20260620 09:25:23.735597 21381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:45411
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:25:23.752065 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:40941
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:23.835776 21577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:23.835978 21577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:23.836002 21577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:23.837096 21577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:23.837131 21577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:23.837173 21577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:25:23.838519 21577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:40941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:23.838698 21577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:23.838861 21577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:23.843773 21583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:23.843796 21577 server_base.cc:1034] running on GCE node
W20260620 09:25:23.843825 21582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:23.843897 21585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:23.844142 21577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:23.844453 21577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:23.845593 21577 hybrid_clock.cc:648] HybridClock initialized: now 1781947523845578 us; error 41 us; skew 500 ppm
Jun 20 09:25:23 dist-test-slave-414w krb5kdc[21336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947523, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:23.848024 21577 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:25:23.848433 21577 webserver.cc:460] Webserver started at http://127.4.59.2:38257/ using document root <none> and password file <none>
I20260620 09:25:23.848647 21577 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:23.848716 21577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:23.848862 21577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:23.849907 21577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "89a432506c684bf38d592d935564dc1d"
format_stamp: "Formatted at 2026-06-20 09:25:23 on dist-test-slave-414w"
server_key: "dff83276271fdc6aaf05a2ef7ce0252d"
server_key_iv: "6c3536cc1aec6e3a8c7c2041b3c14b15"
server_key_version: "encryptionkey@0"
I20260620 09:25:23.850258 21577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "89a432506c684bf38d592d935564dc1d"
format_stamp: "Formatted at 2026-06-20 09:25:23 on dist-test-slave-414w"
server_key: "dff83276271fdc6aaf05a2ef7ce0252d"
server_key_iv: "6c3536cc1aec6e3a8c7c2041b3c14b15"
server_key_version: "encryptionkey@0"
I20260620 09:25:23.851497 21577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:25:23.852144 21592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:23.852311 21577 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:23.852407 21577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "89a432506c684bf38d592d935564dc1d"
format_stamp: "Formatted at 2026-06-20 09:25:23 on dist-test-slave-414w"
server_key: "dff83276271fdc6aaf05a2ef7ce0252d"
server_key_iv: "6c3536cc1aec6e3a8c7c2041b3c14b15"
server_key_version: "encryptionkey@0"
I20260620 09:25:23.852481 21577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:23.870646 21577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:23.872515 21577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:23.872624 21577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:23.872802 21577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:23.873126 21577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:23.873216 21577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:23.873250 21577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:23.873277 21577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:23.879173 21577 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:44835
I20260620 09:25:23.879256 21705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:44835 every 8 connection(s)
I20260620 09:25:23.879577 21577 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
Jun 20 09:25:23 dist-test-slave-414w krb5kdc[21336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947523, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:23.886849 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 21577
I20260620 09:25:23.886967 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:25:23.887156 4332 external_mini_cluster.cc:1428] Setting key f5d2185c0d35f640852f88c556ca0f07
I20260620 09:25:23.889591 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:23.880356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:41687 (local address 127.4.59.62:40941)
0620 09:25:23.880449 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:23.880451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:23.880659 (+ 208us) server_negotiation.cc:407] Connection header received
0620 09:25:23.881250 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:23.881251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:23.881264 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:23.881335 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:23.882310 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:23.882900 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:23.883916 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:23.884132 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:23.884208 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:23.886509 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:23.886511 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:23.886512 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:23.886519 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:23.888160 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:23.888577 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:23.888578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:23.888578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:23.888642 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:23.888893 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:23.888894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:23.888894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:23.889014 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:23.889092 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:23.889421 (+ 329us) server_negotiation.cc:299] Negotiation successful
0620 09:25:23.889502 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 09:25:23.889792 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:23.880281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40941 (local address 127.4.59.2:41687)
0620 09:25:23.880513 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:23.880561 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:23.881098 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:23.881400 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:23.881404 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:23.882001 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:23.882208 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:23.882213 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:23.882991 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:23.882992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:23.883811 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:23.883814 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:23.884212 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:23.884213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:23.884348 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:23.884881 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:23.884890 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:23.886394 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:23.888261 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:23.888268 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:23.888270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:23.888486 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:23.888714 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:23.888715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:23.888715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:23.888824 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:23.889083 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:23.889089 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:23.889281 (+ 192us) client_negotiation.cc:769] Sending connection context
0620 09:25:23.889388 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 09:25:23.889544 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":103,"threads_started":1}
I20260620 09:25:23.890496 21706 heartbeater.cc:347] Connected to a master server at 127.4.59.62:40941
I20260620 09:25:23.890591 21706 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:23.890806 21706 heartbeater.cc:511] Master 127.4.59.62:40941 requested a full tablet report, sending...
I20260620 09:25:23.891191 21381 ts_manager.cc:194] Registered new tserver with Master: 89a432506c684bf38d592d935564dc1d (127.4.59.2:44835)
I20260620 09:25:23.891566 21381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:41687
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:25:23.912393 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:40941
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:23.991011 21711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:23.991232 21711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:23.991271 21711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:23.992384 21711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:23.992446 21711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:23.992491 21711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:25:23.993953 21711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:40941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:23.994233 21711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:23.994448 21711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:23.999364 21717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:23.999461 21711 server_base.cc:1034] running on GCE node
W20260620 09:25:23.999439 21719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:23.999374 21716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:23.999725 21711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:23.999967 21711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:24.001132 21711 hybrid_clock.cc:648] HybridClock initialized: now 1781947524001118 us; error 24 us; skew 500 ppm
Jun 20 09:25:23 dist-test-slave-414w krb5kdc[21336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947523, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:24.003453 21711 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:25:24.003880 21711 webserver.cc:460] Webserver started at http://127.4.59.3:32917/ using document root <none> and password file <none>
I20260620 09:25:24.004062 21711 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:24.004102 21711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:24.004186 21711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:24.005084 21711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "fdb13b82c62c4e85a76f8ea408569ade"
format_stamp: "Formatted at 2026-06-20 09:25:23 on dist-test-slave-414w"
server_key: "ad79d084fa16840d6b57691febb3bd9b"
server_key_iv: "a8cc5463c98693c0857050b54c55664b"
server_key_version: "encryptionkey@0"
I20260620 09:25:24.005439 21711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "fdb13b82c62c4e85a76f8ea408569ade"
format_stamp: "Formatted at 2026-06-20 09:25:23 on dist-test-slave-414w"
server_key: "ad79d084fa16840d6b57691febb3bd9b"
server_key_iv: "a8cc5463c98693c0857050b54c55664b"
server_key_version: "encryptionkey@0"
I20260620 09:25:24.006690 21711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:24.007411 21726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:24.007586 21711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:24.007659 21711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "fdb13b82c62c4e85a76f8ea408569ade"
format_stamp: "Formatted at 2026-06-20 09:25:23 on dist-test-slave-414w"
server_key: "ad79d084fa16840d6b57691febb3bd9b"
server_key_iv: "a8cc5463c98693c0857050b54c55664b"
server_key_version: "encryptionkey@0"
I20260620 09:25:24.007710 21711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:24.021032 21711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:24.022835 21711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:24.022940 21711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:24.023103 21711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:24.023406 21711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:24.023460 21711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:24.023494 21711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:24.023550 21711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:24.029660 21711 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:43993
I20260620 09:25:24.029739 21839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:43993 every 8 connection(s)
I20260620 09:25:24.030049 21711 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
Jun 20 09:25:24 dist-test-slave-414w krb5kdc[21336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947523, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:24.036613 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 21711
I20260620 09:25:24.036717 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:25:24.036892 4332 external_mini_cluster.cc:1428] Setting key 8753faaed03cae27417d4335c19997b1
I20260620 09:25:24.039023 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.030729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:40973 (local address 127.4.59.62:40941)
0620 09:25:24.030808 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.030810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.030972 (+ 162us) server_negotiation.cc:407] Connection header received
0620 09:25:24.031428 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.031429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.031439 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.031497 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:24.032458 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.032943 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.033966 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.034158 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.034214 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.036308 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:24.036309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:24.036309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:24.036315 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:24.037995 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:24.038350 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:24.038351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:24.038351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:24.038401 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:24.038571 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:24.038572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:24.038572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:24.038665 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:24.038713 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.038905 (+ 192us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.038963 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 09:25:24.039172 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.030690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:40941 (local address 127.4.59.3:40973)
0620 09:25:24.030855 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:24.030889 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:24.031343 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:24.031610 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:24.031613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:24.032179 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:24.032374 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.032378 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.033025 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.033026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.033895 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.033898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.034249 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.034250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.034353 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.034881 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:24.034889 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:24.036209 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:24.038060 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:24.038065 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:24.038066 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:24.038284 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:24.038443 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:24.038444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:24.038444 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:24.038523 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:24.038704 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:24.038706 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:24.038804 (+ 98us) client_negotiation.cc:769] Sending connection context
0620 09:25:24.038876 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 09:25:24.038981 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260620 09:25:24.039655 21840 heartbeater.cc:347] Connected to a master server at 127.4.59.62:40941
I20260620 09:25:24.039757 21840 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:24.039916 21840 heartbeater.cc:511] Master 127.4.59.62:40941 requested a full tablet report, sending...
I20260620 09:25:24.040310 21381 ts_manager.cc:194] Registered new tserver with Master: fdb13b82c62c4e85a76f8ea408569ade (127.4.59.3:43993)
I20260620 09:25:24.040673 21381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:40973
I20260620 09:25:24.048826 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:25:24.056042 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.050286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37554 (local address 127.4.59.62:40941)
0620 09:25:24.050440 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.050442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.050459 (+ 17us) server_negotiation.cc:407] Connection header received
0620 09:25:24.050475 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.050476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.050483 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.050536 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:24.051050 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.051500 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.052307 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.052482 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.052550 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.053714 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:24.053715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:24.053715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:24.053721 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:24.055108 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:24.055434 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:24.055435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:24.055435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:24.055486 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:24.055656 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:24.055656 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:24.055656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:24.055780 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:24.055853 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.055933 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.055984 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260620 09:25:24.057560 21381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:25:24.059013 21381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:25:24.064769 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.061111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:36153 (local address 127.0.0.1:59664)
0620 09:25:24.061244 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:24.061288 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:24.061355 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:24.061694 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:24.061697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:24.061700 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:24.061767 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.061769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.062790 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.062791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.063616 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.063618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.064488 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.064489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.064573 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.064588 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 09:25:24.064654 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 09:25:24.064700 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
I20260620 09:25:24.064805 21852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.061246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59664 (local address 127.4.59.1:36153)
0620 09:25:24.061432 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.061437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.061446 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:25:24.061481 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.061483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.061531 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.061606 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:24.061888 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.062688 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.063702 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.064401 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.064478 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.064538 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.064680 (+ 142us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.064723 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":77,"threads_started":1}
I20260620 09:25:24.065560 21853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.061343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:44835 (local address 127.0.0.1:54022)
0620 09:25:24.061661 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:24.061669 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:24.061748 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:24.061997 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:24.061997 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:24.061999 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:24.062041 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.062043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.063313 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.063314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.064438 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.064441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.065318 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.065319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.065392 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.065396 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:25:24.065460 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 09:25:24.065497 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":73,"threads_started":1}
I20260620 09:25:24.065567 21854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.061439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.4.59.2:44835)
0620 09:25:24.061608 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.061612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.061743 (+ 131us) server_negotiation.cc:407] Connection header received
0620 09:25:24.061824 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.061826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.061845 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.061920 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:24.062099 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.063200 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.064525 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.065239 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.065309 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.065361 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.065475 (+ 114us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.065511 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":62,"threads_started":1}
I20260620 09:25:24.065846 21506 tablet_service.cc:1467] Processing CreateTablet for tablet 570e272173b1486c9e028a7ea004eb53 (DEFAULT_TABLE table=test-table [id=d8686606c3344804a5ebc2e4980f7e64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:24.066186 21506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 570e272173b1486c9e028a7ea004eb53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:24.066264 21640 tablet_service.cc:1467] Processing CreateTablet for tablet 570e272173b1486c9e028a7ea004eb53 (DEFAULT_TABLE table=test-table [id=d8686606c3344804a5ebc2e4980f7e64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:24.066560 21640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 570e272173b1486c9e028a7ea004eb53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:24.069598 21857 tablet_bootstrap.cc:492] T 570e272173b1486c9e028a7ea004eb53 P 0d01242fdcce44e7a54d9d1b8b2dfafb: Bootstrap starting.
I20260620 09:25:24.069780 21855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.061509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:43993 (local address 127.0.0.1:48060)
0620 09:25:24.061831 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:24.061840 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:24.061877 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:24.062375 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:24.062376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:24.062378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:24.062420 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.062421 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.063598 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.063600 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.064504 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.064506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.069577 (+ 5071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.069578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.069640 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.069644 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:25:24.069676 (+ 32us) client_negotiation.cc:240] Negotiation successful
0620 09:25:24.069713 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":58,"threads_started":1}
I20260620 09:25:24.070103 21856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.061622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48060 (local address 127.4.59.3:43993)
0620 09:25:24.062163 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.062168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.062177 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:25:24.062221 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.062223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.062238 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.062308 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:24.062489 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.063493 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.064596 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.065354 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.069907 (+ 4553us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.069974 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.069997 (+ 23us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.070041 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":108,"threads_started":1}
I20260620 09:25:24.070338 21857 tablet_bootstrap.cc:654] T 570e272173b1486c9e028a7ea004eb53 P 0d01242fdcce44e7a54d9d1b8b2dfafb: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:24.070730 21774 tablet_service.cc:1467] Processing CreateTablet for tablet 570e272173b1486c9e028a7ea004eb53 (DEFAULT_TABLE table=test-table [id=d8686606c3344804a5ebc2e4980f7e64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:24.070919 21857 log.cc:824] T 570e272173b1486c9e028a7ea004eb53 P 0d01242fdcce44e7a54d9d1b8b2dfafb: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:24.071039 21774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 570e272173b1486c9e028a7ea004eb53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:24.071236 21858 tablet_bootstrap.cc:492] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d: Bootstrap starting.
I20260620 09:25:24.071601 21857 tablet_bootstrap.cc:492] T 570e272173b1486c9e028a7ea004eb53 P 0d01242fdcce44e7a54d9d1b8b2dfafb: No bootstrap required, opened a new log
I20260620 09:25:24.071683 21857 ts_tablet_manager.cc:1397] T 570e272173b1486c9e028a7ea004eb53 P 0d01242fdcce44e7a54d9d1b8b2dfafb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:25:24.071844 21858 tablet_bootstrap.cc:654] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:24.072386 21858 log.cc:824] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:24.073122 21858 tablet_bootstrap.cc:492] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d: No bootstrap required, opened a new log
I20260620 09:25:24.073283 21858 ts_tablet_manager.cc:1397] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:25:24.073472 21857 raft_consensus.cc:348] T 570e272173b1486c9e028a7ea004eb53 P 0d01242fdcce44e7a54d9d1b8b2dfafb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01242fdcce44e7a54d9d1b8b2dfafb" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36153 } } peers { permanent_uuid: "89a432506c684bf38d592d935564dc1d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 44835 } } peers { permanent_uuid: "fdb13b82c62c4e85a76f8ea408569ade" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43993 } }
I20260620 09:25:24.073627 21857 raft_consensus.cc:374] T 570e272173b1486c9e028a7ea004eb53 P 0d01242fdcce44e7a54d9d1b8b2dfafb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:24.073688 21857 raft_consensus.cc:729] T 570e272173b1486c9e028a7ea004eb53 P 0d01242fdcce44e7a54d9d1b8b2dfafb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d01242fdcce44e7a54d9d1b8b2dfafb, State: Initialized, Role: FOLLOWER
I20260620 09:25:24.073809 21857 consensus_queue.cc:260] T 570e272173b1486c9e028a7ea004eb53 P 0d01242fdcce44e7a54d9d1b8b2dfafb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01242fdcce44e7a54d9d1b8b2dfafb" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36153 } } peers { permanent_uuid: "89a432506c684bf38d592d935564dc1d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 44835 } } peers { permanent_uuid: "fdb13b82c62c4e85a76f8ea408569ade" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43993 } }
I20260620 09:25:24.074103 21857 ts_tablet_manager.cc:1428] T 570e272173b1486c9e028a7ea004eb53 P 0d01242fdcce44e7a54d9d1b8b2dfafb: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 09:25:24.074118 21572 heartbeater.cc:503] Master 127.4.59.62:40941 was elected leader, sending a full tablet report...
I20260620 09:25:24.075361 21858 raft_consensus.cc:348] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01242fdcce44e7a54d9d1b8b2dfafb" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36153 } } peers { permanent_uuid: "89a432506c684bf38d592d935564dc1d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 44835 } } peers { permanent_uuid: "fdb13b82c62c4e85a76f8ea408569ade" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43993 } }
I20260620 09:25:24.075487 21858 raft_consensus.cc:374] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:24.075510 21858 raft_consensus.cc:729] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89a432506c684bf38d592d935564dc1d, State: Initialized, Role: FOLLOWER
I20260620 09:25:24.075606 21858 consensus_queue.cc:260] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01242fdcce44e7a54d9d1b8b2dfafb" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36153 } } peers { permanent_uuid: "89a432506c684bf38d592d935564dc1d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 44835 } } peers { permanent_uuid: "fdb13b82c62c4e85a76f8ea408569ade" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43993 } }
I20260620 09:25:24.075820 21858 ts_tablet_manager.cc:1428] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:25:24.075882 21706 heartbeater.cc:503] Master 127.4.59.62:40941 was elected leader, sending a full tablet report...
I20260620 09:25:24.076010 21862 tablet_bootstrap.cc:492] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade: Bootstrap starting.
I20260620 09:25:24.076658 21862 tablet_bootstrap.cc:654] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:24.077126 21862 log.cc:824] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:24.077811 21862 tablet_bootstrap.cc:492] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade: No bootstrap required, opened a new log
I20260620 09:25:24.077883 21862 ts_tablet_manager.cc:1397] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 09:25:24.079152 21862 raft_consensus.cc:348] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01242fdcce44e7a54d9d1b8b2dfafb" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36153 } } peers { permanent_uuid: "89a432506c684bf38d592d935564dc1d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 44835 } } peers { permanent_uuid: "fdb13b82c62c4e85a76f8ea408569ade" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43993 } }
I20260620 09:25:24.079275 21862 raft_consensus.cc:374] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:24.079298 21862 raft_consensus.cc:729] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdb13b82c62c4e85a76f8ea408569ade, State: Initialized, Role: FOLLOWER
I20260620 09:25:24.079424 21862 consensus_queue.cc:260] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01242fdcce44e7a54d9d1b8b2dfafb" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36153 } } peers { permanent_uuid: "89a432506c684bf38d592d935564dc1d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 44835 } } peers { permanent_uuid: "fdb13b82c62c4e85a76f8ea408569ade" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43993 } }
I20260620 09:25:24.079685 21862 ts_tablet_manager.cc:1428] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 09:25:24.079697 21840 heartbeater.cc:503] Master 127.4.59.62:40941 was elected leader, sending a full tablet report...
W20260620 09:25:24.130089 21707 tablet.cc:2369] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:25:24.190137 21863 raft_consensus.cc:482] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:25:24.190271 21863 raft_consensus.cc:504] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01242fdcce44e7a54d9d1b8b2dfafb" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36153 } } peers { permanent_uuid: "89a432506c684bf38d592d935564dc1d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 44835 } } peers { permanent_uuid: "fdb13b82c62c4e85a76f8ea408569ade" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43993 } }
I20260620 09:25:24.190573 21863 leader_election.cc:288] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d01242fdcce44e7a54d9d1b8b2dfafb (127.4.59.1:36153), fdb13b82c62c4e85a76f8ea408569ade (127.4.59.3:43993)
I20260620 09:25:24.194378 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.190769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:36153 (local address 127.4.59.2:56197)
0620 09:25:24.190824 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:24.190837 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:24.190880 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:24.191047 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:24.191048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:24.191050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:24.191118 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.191120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.192082 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.192082 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.193296 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.193299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.194084 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.194085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.194171 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.194172 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 09:25:24.194234 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 09:25:24.194278 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":5}
I20260620 09:25:24.194392 21852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.190819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:56197 (local address 127.4.59.1:36153)
0620 09:25:24.190864 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.190868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.190931 (+ 63us) server_negotiation.cc:407] Connection header received
0620 09:25:24.190963 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.190964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.190976 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.191039 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:24.191219 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.192011 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.193410 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.193972 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.194077 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.194087 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.194283 (+ 196us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.194326 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 09:25:24.194392 21856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.190977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:52583 (local address 127.4.59.3:43993)
0620 09:25:24.191045 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.191047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.191200 (+ 153us) server_negotiation.cc:407] Connection header received
0620 09:25:24.191282 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.191283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.191295 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.191347 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:24.191484 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.192151 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.193410 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.194003 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.194077 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.194087 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.194283 (+ 196us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.194326 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 09:25:24.194386 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.190895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:43993 (local address 127.4.59.2:52583)
0620 09:25:24.191115 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:24.191123 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:24.191224 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:24.191360 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:24.191360 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:24.191361 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:24.191416 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.191416 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.192236 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.192237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.193317 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.193319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.194085 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.194085 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.194167 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.194171 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:25:24.194244 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 09:25:24.194278 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":137,"threads_started":1}
I20260620 09:25:24.194851 21526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "570e272173b1486c9e028a7ea004eb53" candidate_uuid: "89a432506c684bf38d592d935564dc1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d01242fdcce44e7a54d9d1b8b2dfafb" is_pre_election: true
I20260620 09:25:24.194835 21794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "570e272173b1486c9e028a7ea004eb53" candidate_uuid: "89a432506c684bf38d592d935564dc1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdb13b82c62c4e85a76f8ea408569ade" is_pre_election: true
I20260620 09:25:24.194974 21794 raft_consensus.cc:2454] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89a432506c684bf38d592d935564dc1d in term 0.
I20260620 09:25:24.195021 21526 raft_consensus.cc:2454] T 570e272173b1486c9e028a7ea004eb53 P 0d01242fdcce44e7a54d9d1b8b2dfafb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89a432506c684bf38d592d935564dc1d in term 0.
I20260620 09:25:24.195235 21596 leader_election.cc:302] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89a432506c684bf38d592d935564dc1d, fdb13b82c62c4e85a76f8ea408569ade; no voters:
I20260620 09:25:24.195364 21863 raft_consensus.cc:2781] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:25:24.195456 21863 raft_consensus.cc:482] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:25:24.195489 21863 raft_consensus.cc:3037] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:24.196233 21863 raft_consensus.cc:504] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01242fdcce44e7a54d9d1b8b2dfafb" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36153 } } peers { permanent_uuid: "89a432506c684bf38d592d935564dc1d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 44835 } } peers { permanent_uuid: "fdb13b82c62c4e85a76f8ea408569ade" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43993 } }
I20260620 09:25:24.196368 21863 leader_election.cc:288] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [CANDIDATE]: Term 1 election: Requested vote from peers 0d01242fdcce44e7a54d9d1b8b2dfafb (127.4.59.1:36153), fdb13b82c62c4e85a76f8ea408569ade (127.4.59.3:43993)
I20260620 09:25:24.196553 21794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "570e272173b1486c9e028a7ea004eb53" candidate_uuid: "89a432506c684bf38d592d935564dc1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdb13b82c62c4e85a76f8ea408569ade"
I20260620 09:25:24.196547 21526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "570e272173b1486c9e028a7ea004eb53" candidate_uuid: "89a432506c684bf38d592d935564dc1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d01242fdcce44e7a54d9d1b8b2dfafb"
I20260620 09:25:24.196620 21794 raft_consensus.cc:3037] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:24.196638 21526 raft_consensus.cc:3037] T 570e272173b1486c9e028a7ea004eb53 P 0d01242fdcce44e7a54d9d1b8b2dfafb [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:24.197453 21794 raft_consensus.cc:2454] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89a432506c684bf38d592d935564dc1d in term 1.
I20260620 09:25:24.197461 21526 raft_consensus.cc:2454] T 570e272173b1486c9e028a7ea004eb53 P 0d01242fdcce44e7a54d9d1b8b2dfafb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89a432506c684bf38d592d935564dc1d in term 1.
I20260620 09:25:24.197602 21596 leader_election.cc:302] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89a432506c684bf38d592d935564dc1d, fdb13b82c62c4e85a76f8ea408569ade; no voters:
I20260620 09:25:24.197741 21863 raft_consensus.cc:2781] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:24.197889 21863 raft_consensus.cc:686] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [term 1 LEADER]: Becoming Leader. State: Replica: 89a432506c684bf38d592d935564dc1d, State: Running, Role: LEADER
I20260620 09:25:24.197989 21863 consensus_queue.cc:237] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01242fdcce44e7a54d9d1b8b2dfafb" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36153 } } peers { permanent_uuid: "89a432506c684bf38d592d935564dc1d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 44835 } } peers { permanent_uuid: "fdb13b82c62c4e85a76f8ea408569ade" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43993 } }
I20260620 09:25:24.198729 21380 catalog_manager.cc:5495] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d reported cstate change: term changed from 0 to 1, leader changed from <none> to 89a432506c684bf38d592d935564dc1d (127.4.59.2). New cstate: current_term: 1 leader_uuid: "89a432506c684bf38d592d935564dc1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01242fdcce44e7a54d9d1b8b2dfafb" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89a432506c684bf38d592d935564dc1d" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 44835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fdb13b82c62c4e85a76f8ea408569ade" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260620 09:25:24.212925 21854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.210644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54026 (local address 127.4.59.2:44835)
0620 09:25:24.210701 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.210703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.210714 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:25:24.210746 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.210747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.210766 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.210849 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:25:24.210994 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.211445 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.212185 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.212377 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.212460 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.212627 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:25:24.212725 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:25:24.212816 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.212835 (+ 19us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.212854 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 09:25:24.214584 21794 raft_consensus.cc:1261] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade [term 1 FOLLOWER]: Refusing update from remote peer 89a432506c684bf38d592d935564dc1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:25:24.214592 21526 raft_consensus.cc:1261] T 570e272173b1486c9e028a7ea004eb53 P 0d01242fdcce44e7a54d9d1b8b2dfafb [term 1 FOLLOWER]: Refusing update from remote peer 89a432506c684bf38d592d935564dc1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:25:24.214908 21863 consensus_queue.cc:1035] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d01242fdcce44e7a54d9d1b8b2dfafb" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:25:24.215044 21867 consensus_queue.cc:1035] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdb13b82c62c4e85a76f8ea408569ade" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:25:24.217185 21870 mvcc.cc:205] Tried to move back new op lower bound from 7298857059180675072 to 7298857059115601920. Current Snapshot: MvccSnapshot[applied={T|T < 7298857059180675072}]
I20260620 09:25:24.218211 21872 mvcc.cc:205] Tried to move back new op lower bound from 7298857059180675072 to 7298857059115601920. Current Snapshot: MvccSnapshot[applied={T|T < 7298857059180675072}]
I20260620 09:25:24.219552 21381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:25:24.219635 21381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:25:24.220317 21381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21ec2802e5be4831985f32b8ac95dcac: Sending DeleteTablet for 3 replicas of tablet 570e272173b1486c9e028a7ea004eb53
I20260620 09:25:24.220633 21506 tablet_service.cc:1514] Processing DeleteTablet for tablet 570e272173b1486c9e028a7ea004eb53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:24 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:59664
I20260620 09:25:24.220638 21640 tablet_service.cc:1514] Processing DeleteTablet for tablet 570e272173b1486c9e028a7ea004eb53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:24 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:54022
I20260620 09:25:24.220791 21774 tablet_service.cc:1514] Processing DeleteTablet for tablet 570e272173b1486c9e028a7ea004eb53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:24 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:48060
I20260620 09:25:24.220933 21881 tablet_replica.cc:330] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d: stopping tablet replica
I20260620 09:25:24.221028 21881 raft_consensus.cc:2229] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:25:24.221060 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 21443
I20260620 09:25:24.221115 21881 raft_consensus.cc:2258] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:25:24.222287 21881 ts_tablet_manager.cc:1905] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:25:24.223701 21882 tablet_replica.cc:330] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade: stopping tablet replica
I20260620 09:25:24.223796 21882 raft_consensus.cc:2229] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:25:24.223887 21882 raft_consensus.cc:2258] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:25:24.224126 21881 ts_tablet_manager.cc:1918] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:25:24.224200 21881 log.cc:1196] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/570e272173b1486c9e028a7ea004eb53
I20260620 09:25:24.224444 21882 ts_tablet_manager.cc:1905] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:25:24.224465 21881 ts_tablet_manager.cc:1939] T 570e272173b1486c9e028a7ea004eb53 P 89a432506c684bf38d592d935564dc1d: Deleting consensus metadata
I20260620 09:25:24.225471 21368 catalog_manager.cc:4841] TS 89a432506c684bf38d592d935564dc1d (127.4.59.2:44835): tablet 570e272173b1486c9e028a7ea004eb53 (table test-table [id=d8686606c3344804a5ebc2e4980f7e64]) successfully deleted
I20260620 09:25:24.226169 21882 ts_tablet_manager.cc:1918] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:25:24.226295 21882 log.cc:1196] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/570e272173b1486c9e028a7ea004eb53
I20260620 09:25:24.226631 21882 ts_tablet_manager.cc:1939] T 570e272173b1486c9e028a7ea004eb53 P fdb13b82c62c4e85a76f8ea408569ade: Deleting consensus metadata
I20260620 09:25:24.227061 21368 catalog_manager.cc:4841] TS fdb13b82c62c4e85a76f8ea408569ade (127.4.59.3:43993): tablet 570e272173b1486c9e028a7ea004eb53 (table test-table [id=d8686606c3344804a5ebc2e4980f7e64]) successfully deleted
W20260620 09:25:24.227483 21368 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.59.1:36153 (error 108)
I20260620 09:25:24.227752 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 21577
I20260620 09:25:24.228323 21855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.227838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:36153 (local address 127.0.0.1:59672)
0620 09:25:24.228247 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:24.228285 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391}
W20260620 09:25:24.228765 21368 catalog_manager.cc:4584] TS 0d01242fdcce44e7a54d9d1b8b2dfafb (127.4.59.1:36153): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 570e272173b1486c9e028a7ea004eb53: Network error: Client connection negotiation failed: client connection to 127.4.59.1:36153: connect: Connection refused (error 111)
I20260620 09:25:24.232640 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 21711
I20260620 09:25:24.236923 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 21349
2026-06-20T09:25:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1987 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-20T09:25:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:24Z Disabled control of system clock
I20260620 09:25:24.264292 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:32813
--webserver_interface=127.4.59.62
--webserver_port=0
--builtin_ntp_servers=127.4.59.20:42625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:32813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 09:25:24.343631 21887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:24.343835 21887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:24.343855 21887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:24.344990 21887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:24.345031 21887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:24.345043 21887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:24.345055 21887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:24.345067 21887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:24.346490 21887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:32813
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:32813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:24.346746 21887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:24.347050 21887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:24.352015 21892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:24.352236 21887 server_base.cc:1034] running on GCE node
W20260620 09:25:24.351997 21893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:24.352015 21895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:24.352571 21887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:24.352891 21887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:24.354028 21887 hybrid_clock.cc:648] HybridClock initialized: now 1781947524354013 us; error 36 us; skew 500 ppm
I20260620 09:25:24.355207 21887 webserver.cc:460] Webserver started at http://127.4.59.62:41361/ using document root <none> and password file <none>
I20260620 09:25:24.355458 21887 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:24.355526 21887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:24.355670 21887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:24.356707 21887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "2e357f79099e43c5ba8397e104e1fd23"
format_stamp: "Formatted at 2026-06-20 09:25:24 on dist-test-slave-414w"
server_key: "ff45d054d375f0bcc0e0a3660bf2ef1a"
server_key_iv: "814ac126a4cd40d8f69b6cf685f27b7f"
server_key_version: "encryptionkey@0"
I20260620 09:25:24.357064 21887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "2e357f79099e43c5ba8397e104e1fd23"
format_stamp: "Formatted at 2026-06-20 09:25:24 on dist-test-slave-414w"
server_key: "ff45d054d375f0bcc0e0a3660bf2ef1a"
server_key_iv: "814ac126a4cd40d8f69b6cf685f27b7f"
server_key_version: "encryptionkey@0"
I20260620 09:25:24.358369 21887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:25:24.359062 21901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:24.359280 21887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:25:24.359376 21887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "2e357f79099e43c5ba8397e104e1fd23"
format_stamp: "Formatted at 2026-06-20 09:25:24 on dist-test-slave-414w"
server_key: "ff45d054d375f0bcc0e0a3660bf2ef1a"
server_key_iv: "814ac126a4cd40d8f69b6cf685f27b7f"
server_key_version: "encryptionkey@0"
I20260620 09:25:24.359458 21887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:24.384699 21887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:24.385011 21887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:24.385214 21887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:24.389240 21887 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:32813
I20260620 09:25:24.389283 21953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:32813 every 8 connection(s)
I20260620 09:25:24.389605 21887 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:24.390468 21954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:24.392645 21954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23: Bootstrap starting.
I20260620 09:25:24.393257 21954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:24.393538 21954 log.cc:824] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:24.394207 21954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23: No bootstrap required, opened a new log
I20260620 09:25:24.395717 21954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e357f79099e43c5ba8397e104e1fd23" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 32813 } }
I20260620 09:25:24.395874 21954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:24.395920 21954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e357f79099e43c5ba8397e104e1fd23, State: Initialized, Role: FOLLOWER
I20260620 09:25:24.396085 21954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e357f79099e43c5ba8397e104e1fd23" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 32813 } }
I20260620 09:25:24.396193 21954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:24.396236 21954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:24.396284 21954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:24.396772 21954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e357f79099e43c5ba8397e104e1fd23" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 32813 } }
I20260620 09:25:24.396886 21954 leader_election.cc:302] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e357f79099e43c5ba8397e104e1fd23; no voters:
I20260620 09:25:24.397061 21954 leader_election.cc:288] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:24.397109 21957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:24.397333 21954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:24.397368 21957 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [term 1 LEADER]: Becoming Leader. State: Replica: 2e357f79099e43c5ba8397e104e1fd23, State: Running, Role: LEADER
I20260620 09:25:24.397460 21957 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e357f79099e43c5ba8397e104e1fd23" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 32813 } }
I20260620 09:25:24.397861 21959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e357f79099e43c5ba8397e104e1fd23. Latest consensus state: current_term: 1 leader_uuid: "2e357f79099e43c5ba8397e104e1fd23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e357f79099e43c5ba8397e104e1fd23" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 32813 } } }
I20260620 09:25:24.397940 21959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:24.397948 21958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e357f79099e43c5ba8397e104e1fd23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e357f79099e43c5ba8397e104e1fd23" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 32813 } } }
I20260620 09:25:24.398161 21958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:24.398149 21966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:24.398804 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 21887
I20260620 09:25:24.398911 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:24.399083 4332 external_mini_cluster.cc:1428] Setting key d56ffa7ef95fda96eaca894c21d8c530
I20260620 09:25:24.399323 21966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:24.400800 21966 catalog_manager.cc:1347] Generated new cluster ID: ed4919e7d527410f90019e21d2686f61
I20260620 09:25:24.400856 21966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:24.410503 21976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.399660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52844 (local address 127.4.59.62:32813)
0620 09:25:24.399842 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.399852 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.405406 (+ 5554us) server_negotiation.cc:407] Connection header received
0620 09:25:24.405558 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.405565 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.405631 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.405757 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:24.405981 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.407338 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.408450 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.409027 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.409131 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.409849 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:24.409853 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:24.409853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:24.410043 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:24.410081 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:24.410095 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:24.410154 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.410232 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.410302 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260620 09:25:24.417073 21966 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:24.417598 21966 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:24.419783 21966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23: Generated new TSK 0
I20260620 09:25:24.419946 21966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:24.424283 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:32813
--builtin_ntp_servers=127.4.59.20:42625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 09:25:24.518510 21978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:24.518694 21978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:24.518715 21978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:24.519836 21978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:24.519873 21978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:24.519898 21978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:25:24.521227 21978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:32813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:24.521448 21978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:24.521744 21978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:24.526383 21986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:24.526409 21983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:24.526401 21984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:24.526674 21978 server_base.cc:1034] running on GCE node
I20260620 09:25:24.526882 21978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:24.527129 21978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:24.528316 21978 hybrid_clock.cc:648] HybridClock initialized: now 1781947524528302 us; error 42 us; skew 500 ppm
I20260620 09:25:24.529748 21978 webserver.cc:460] Webserver started at http://127.4.59.1:37649/ using document root <none> and password file <none>
I20260620 09:25:24.530000 21978 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:24.530054 21978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:24.530179 21978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:24.531443 21978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "36d10f923f3a4587b1c5520ab53ad110"
format_stamp: "Formatted at 2026-06-20 09:25:24 on dist-test-slave-414w"
server_key: "f4fbc6ace2a7097f11ce3d940de88fa5"
server_key_iv: "0b3532eee339f8233704f74b91154f9b"
server_key_version: "encryptionkey@0"
I20260620 09:25:24.531813 21978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "36d10f923f3a4587b1c5520ab53ad110"
format_stamp: "Formatted at 2026-06-20 09:25:24 on dist-test-slave-414w"
server_key: "f4fbc6ace2a7097f11ce3d940de88fa5"
server_key_iv: "0b3532eee339f8233704f74b91154f9b"
server_key_version: "encryptionkey@0"
I20260620 09:25:24.533288 21978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260620 09:25:24.534040 21992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:24.534209 21978 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:24.534303 21978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "36d10f923f3a4587b1c5520ab53ad110"
format_stamp: "Formatted at 2026-06-20 09:25:24 on dist-test-slave-414w"
server_key: "f4fbc6ace2a7097f11ce3d940de88fa5"
server_key_iv: "0b3532eee339f8233704f74b91154f9b"
server_key_version: "encryptionkey@0"
I20260620 09:25:24.534380 21978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:24.545394 21978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:24.545701 21978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:24.545851 21978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:24.546056 21978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:24.546350 21978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:24.546404 21978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:24.546444 21978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:24.546471 21978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:24.551927 21978 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:46709
I20260620 09:25:24.552022 22105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:46709 every 8 connection(s)
I20260620 09:25:24.552299 21978 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:25:24.557123 21976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.553006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:46835 (local address 127.4.59.62:32813)
0620 09:25:24.553053 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.553055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.553462 (+ 407us) server_negotiation.cc:407] Connection header received
0620 09:25:24.553899 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.553900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.553910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.553971 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:24.554550 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.555017 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.555939 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.556119 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.556167 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.556904 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:24.556905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:24.556905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:24.556908 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:24.556920 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:24.556923 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:24.556965 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.557041 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.557063 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 09:25:24.557425 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.553007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:32813 (local address 127.4.59.1:46835)
0620 09:25:24.553354 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:24.553385 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:24.553823 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:24.554020 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:24.554023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:24.554030 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:24.554485 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.554489 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.555106 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.555106 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.555876 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.555878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.556189 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.556189 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.556320 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.556785 (+ 465us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:25:24.556785 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:24.556811 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:25:24.556831 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:24.556976 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:24.556980 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:24.556982 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:25:24.557030 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 09:25:24.557117 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":95,"threads_started":1}
I20260620 09:25:24.557893 22106 heartbeater.cc:347] Connected to a master server at 127.4.59.62:32813
I20260620 09:25:24.557974 22106 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:24.558112 22106 heartbeater.cc:511] Master 127.4.59.62:32813 requested a full tablet report, sending...
I20260620 09:25:24.558557 21918 ts_manager.cc:194] Registered new tserver with Master: 36d10f923f3a4587b1c5520ab53ad110 (127.4.59.1:46709)
I20260620 09:25:24.559160 21918 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.59.1:46835
I20260620 09:25:24.559473 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 21978
I20260620 09:25:24.559572 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:25:24.559741 4332 external_mini_cluster.cc:1428] Setting key ded1ec86c88d23553be417be27c2a58f
I20260620 09:25:24.560560 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:32813
--builtin_ntp_servers=127.4.59.20:42625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 09:25:24.640439 22109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:24.640762 22109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:24.640800 22109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:24.642297 22109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:24.642362 22109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:24.642405 22109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:25:24.643734 22109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:32813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:24.644004 22109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:24.644313 22109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:24.649376 22114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:24.649439 22109 server_base.cc:1034] running on GCE node
W20260620 09:25:24.649565 22115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:24.649598 22117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:24.649822 22109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:24.650091 22109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:24.651263 22109 hybrid_clock.cc:648] HybridClock initialized: now 1781947524651251 us; error 42 us; skew 500 ppm
I20260620 09:25:24.652577 22109 webserver.cc:460] Webserver started at http://127.4.59.2:34831/ using document root <none> and password file <none>
I20260620 09:25:24.652832 22109 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:24.652904 22109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:24.653069 22109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:24.654145 22109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "54ebc32f4a824c85924f1f01f375118b"
format_stamp: "Formatted at 2026-06-20 09:25:24 on dist-test-slave-414w"
server_key: "a94beb7a8c951a91b8d27f0347b89671"
server_key_iv: "c046de90b4c9cdb0364e0953a73990a6"
server_key_version: "encryptionkey@0"
I20260620 09:25:24.654510 22109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "54ebc32f4a824c85924f1f01f375118b"
format_stamp: "Formatted at 2026-06-20 09:25:24 on dist-test-slave-414w"
server_key: "a94beb7a8c951a91b8d27f0347b89671"
server_key_iv: "c046de90b4c9cdb0364e0953a73990a6"
server_key_version: "encryptionkey@0"
I20260620 09:25:24.655761 22109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:25:24.656467 22123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:24.656682 22109 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:24.656760 22109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "54ebc32f4a824c85924f1f01f375118b"
format_stamp: "Formatted at 2026-06-20 09:25:24 on dist-test-slave-414w"
server_key: "a94beb7a8c951a91b8d27f0347b89671"
server_key_iv: "c046de90b4c9cdb0364e0953a73990a6"
server_key_version: "encryptionkey@0"
I20260620 09:25:24.656813 22109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:24.669669 22109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:24.669914 22109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:24.670009 22109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:24.670194 22109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:24.670457 22109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:24.670486 22109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:24.670508 22109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:24.670521 22109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:24.675846 22109 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:34807
I20260620 09:25:24.675921 22236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:34807 every 8 connection(s)
I20260620 09:25:24.676234 22109 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:25:24.681470 21976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.676933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:58497 (local address 127.4.59.62:32813)
0620 09:25:24.676978 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.676979 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.677288 (+ 309us) server_negotiation.cc:407] Connection header received
0620 09:25:24.677797 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.677798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.677810 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.677856 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:24.678530 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.679029 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.680006 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.680208 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.680285 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.681164 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:24.681166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:24.681166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:24.681171 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:24.681185 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:24.681190 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:24.681257 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.681360 (+ 103us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.681392 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 09:25:24.681664 22239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.676921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:32813 (local address 127.4.59.2:58497)
0620 09:25:24.677123 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:24.677193 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:24.677689 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:24.677934 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:24.677938 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:24.677947 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:24.678433 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.678437 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.679129 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.679131 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.679911 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.679914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.680292 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.680294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.680428 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.680978 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:25:24.680979 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:24.681006 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:25:24.681027 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:24.681271 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:24.681276 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:24.681279 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:25:24.681363 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 09:25:24.681461 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260620 09:25:24.682174 22237 heartbeater.cc:347] Connected to a master server at 127.4.59.62:32813
I20260620 09:25:24.682260 22237 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:24.682395 22237 heartbeater.cc:511] Master 127.4.59.62:32813 requested a full tablet report, sending...
I20260620 09:25:24.682739 21918 ts_manager.cc:194] Registered new tserver with Master: 54ebc32f4a824c85924f1f01f375118b (127.4.59.2:34807)
I20260620 09:25:24.683055 21918 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.59.2:58497
I20260620 09:25:24.685340 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 22109
I20260620 09:25:24.685431 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:25:24.685606 4332 external_mini_cluster.cc:1428] Setting key 8361c150a6bf30bb92f855296d92bc5b
I20260620 09:25:24.686475 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:32813
--builtin_ntp_servers=127.4.59.20:42625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 09:25:24.767475 22240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:24.767664 22240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:24.767688 22240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:24.768795 22240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:24.768833 22240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:24.768858 22240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:25:24.770268 22240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:42625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:32813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:24.770493 22240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:24.770776 22240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:24.775576 22248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:24.775620 22246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:24.775846 22240 server_base.cc:1034] running on GCE node
W20260620 09:25:24.775585 22245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:24.776090 22240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:24.776328 22240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:24.777482 22240 hybrid_clock.cc:648] HybridClock initialized: now 1781947524777470 us; error 26 us; skew 500 ppm
I20260620 09:25:24.778609 22240 webserver.cc:460] Webserver started at http://127.4.59.3:40639/ using document root <none> and password file <none>
I20260620 09:25:24.778859 22240 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:24.778924 22240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:24.779062 22240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:24.780048 22240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "60b9f8f5c35c450387355efb92b1d089"
format_stamp: "Formatted at 2026-06-20 09:25:24 on dist-test-slave-414w"
server_key: "e4a42157b9651d8e57e64ce28d5ebac0"
server_key_iv: "ccc733e745fde1bfae5a7d84aa716e2c"
server_key_version: "encryptionkey@0"
I20260620 09:25:24.780407 22240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "60b9f8f5c35c450387355efb92b1d089"
format_stamp: "Formatted at 2026-06-20 09:25:24 on dist-test-slave-414w"
server_key: "e4a42157b9651d8e57e64ce28d5ebac0"
server_key_iv: "ccc733e745fde1bfae5a7d84aa716e2c"
server_key_version: "encryptionkey@0"
I20260620 09:25:24.781754 22240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:25:24.782390 22254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:24.782562 22240 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:24.782627 22240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "60b9f8f5c35c450387355efb92b1d089"
format_stamp: "Formatted at 2026-06-20 09:25:24 on dist-test-slave-414w"
server_key: "e4a42157b9651d8e57e64ce28d5ebac0"
server_key_iv: "ccc733e745fde1bfae5a7d84aa716e2c"
server_key_version: "encryptionkey@0"
I20260620 09:25:24.782675 22240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:24.809300 22240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:24.809545 22240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:24.809643 22240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:24.809814 22240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:24.810137 22240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:24.810195 22240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:24.810232 22240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:24.810281 22240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:24.815754 22240 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:39675
I20260620 09:25:24.815826 22367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:39675 every 8 connection(s)
I20260620 09:25:24.816148 22240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:25:24.821063 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 22240
I20260620 09:25:24.821177 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:25:24.821367 4332 external_mini_cluster.cc:1428] Setting key ce8e0b7d934f37a47dcc66c8a77490ea
I20260620 09:25:24.821470 21976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.816867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:58441 (local address 127.4.59.62:32813)
0620 09:25:24.816909 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.816911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.817119 (+ 208us) server_negotiation.cc:407] Connection header received
0620 09:25:24.817639 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.817640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.817649 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.817715 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:24.818320 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.818807 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.820135 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.820345 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.820401 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.821194 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:24.821196 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:24.821196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:24.821201 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:24.821215 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:24.821219 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:24.821267 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.821365 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.821392 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:25:24.821796 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.816828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:32813 (local address 127.4.59.3:58441)
0620 09:25:24.816996 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:24.817034 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:24.817554 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:24.817770 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:24.817773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:24.817781 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:24.818241 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.818245 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.818879 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.818880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.820038 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.820041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.820416 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.820416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.820519 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.820983 (+ 464us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:25:24.820984 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:24.821023 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:25:24.821052 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:24.821281 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:24.821287 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:24.821290 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:25:24.821365 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 09:25:24.821465 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260620 09:25:24.822197 22368 heartbeater.cc:347] Connected to a master server at 127.4.59.62:32813
I20260620 09:25:24.822288 22368 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:24.822456 22368 heartbeater.cc:511] Master 127.4.59.62:32813 requested a full tablet report, sending...
I20260620 09:25:24.822780 21918 ts_manager.cc:194] Registered new tserver with Master: 60b9f8f5c35c450387355efb92b1d089 (127.4.59.3:39675)
I20260620 09:25:24.823138 21918 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.59.3:58441
I20260620 09:25:24.833056 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:25:24.836959 21976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.834368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52856 (local address 127.4.59.62:32813)
0620 09:25:24.834430 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.834432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.834483 (+ 51us) server_negotiation.cc:407] Connection header received
0620 09:25:24.834532 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.834533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.834544 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.834591 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:24.834745 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.835203 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.835881 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.836056 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.836102 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.836739 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:24.836740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:24.836740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:24.836744 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:24.836757 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:24.836760 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:24.836801 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.836846 (+ 45us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.836866 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 09:25:24.838346 21918 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:25:24.839848 21918 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:25:24.846081 22376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.841826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:46709 (local address 127.0.0.1:55824)
0620 09:25:24.842113 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:24.842124 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:24.842179 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:24.842652 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:24.842653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:24.842655 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:24.842710 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.842715 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.843566 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.843567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.844670 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.844672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.845710 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.845711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.845791 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.845802 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 09:25:24.845875 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 09:25:24.845966 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":36,"thread_start_us":63,"threads_started":1}
I20260620 09:25:24.846204 22378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.841843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:34807 (local address 127.0.0.1:47916)
0620 09:25:24.842286 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:24.842295 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:24.842340 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:24.842634 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:24.842637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:24.842640 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:24.842705 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.842707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.844030 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.844031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.845030 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.845032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.845715 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.845716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.845790 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.845802 (+ 12us) client_negotiation.cc:769] Sending connection context
0620 09:25:24.845873 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 09:25:24.845951 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":2,"thread_start_us":110,"threads_started":1}
I20260620 09:25:24.846349 22377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.841826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:39675 (local address 127.0.0.1:58622)
0620 09:25:24.841993 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:24.842029 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:24.842113 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:24.842656 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:24.842656 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:24.842658 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:24.842710 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.842711 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.844063 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.844064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.845079 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.845081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.845989 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.845989 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.846217 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.846223 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 09:25:24.846255 (+ 32us) client_negotiation.cc:240] Negotiation successful
0620 09:25:24.846292 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":5,"thread_start_us":86,"threads_started":1}
I20260620 09:25:24.846624 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.841984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58622 (local address 127.4.59.3:39675)
0620 09:25:24.842491 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.842495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.842504 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:25:24.842546 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.842548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.842561 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.842781 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:24.842836 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.843942 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.845199 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.845903 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.846432 (+ 529us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.846497 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.846517 (+ 20us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.846553 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":69,"threads_started":1}
I20260620 09:25:24.846623 22381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.842109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47916 (local address 127.4.59.2:34807)
0620 09:25:24.842366 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.842371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.842383 (+ 12us) server_negotiation.cc:407] Connection header received
0620 09:25:24.842424 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.842426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.842442 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.842522 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:24.842788 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.843946 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.845124 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.845614 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.845711 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.845800 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.846063 (+ 263us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.846110 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":186,"threads_started":1}
I20260620 09:25:24.846881 22380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.842138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55824 (local address 127.4.59.1:46709)
0620 09:25:24.842423 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.842427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.842434 (+ 7us) server_negotiation.cc:407] Connection header received
0620 09:25:24.842490 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.842492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.842506 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.842589 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:24.842782 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.843479 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.844741 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.845520 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.845589 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.845674 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.846754 (+ 1080us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.846804 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":68,"threads_started":1}
I20260620 09:25:24.847436 22302 tablet_service.cc:1467] Processing CreateTablet for tablet a5765dcbfb724e179eeed7c1449b2664 (DEFAULT_TABLE table=test-table [id=eaa5b9f4291947979aeac424439f4993]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:24.847455 22171 tablet_service.cc:1467] Processing CreateTablet for tablet a5765dcbfb724e179eeed7c1449b2664 (DEFAULT_TABLE table=test-table [id=eaa5b9f4291947979aeac424439f4993]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:24.847450 22040 tablet_service.cc:1467] Processing CreateTablet for tablet a5765dcbfb724e179eeed7c1449b2664 (DEFAULT_TABLE table=test-table [id=eaa5b9f4291947979aeac424439f4993]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:24.847782 22302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5765dcbfb724e179eeed7c1449b2664. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:24.847784 22171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5765dcbfb724e179eeed7c1449b2664. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:24.847783 22040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5765dcbfb724e179eeed7c1449b2664. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:24.850648 22382 tablet_bootstrap.cc:492] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089: Bootstrap starting.
I20260620 09:25:24.851043 22383 tablet_bootstrap.cc:492] T a5765dcbfb724e179eeed7c1449b2664 P 36d10f923f3a4587b1c5520ab53ad110: Bootstrap starting.
I20260620 09:25:24.851428 22382 tablet_bootstrap.cc:654] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:24.851687 22383 tablet_bootstrap.cc:654] T a5765dcbfb724e179eeed7c1449b2664 P 36d10f923f3a4587b1c5520ab53ad110: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:24.851738 22382 log.cc:824] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:24.852018 22383 log.cc:824] T a5765dcbfb724e179eeed7c1449b2664 P 36d10f923f3a4587b1c5520ab53ad110: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:24.852310 22384 tablet_bootstrap.cc:492] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b: Bootstrap starting.
I20260620 09:25:24.852592 22382 tablet_bootstrap.cc:492] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089: No bootstrap required, opened a new log
I20260620 09:25:24.852691 22382 ts_tablet_manager.cc:1397] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:25:24.852929 22384 tablet_bootstrap.cc:654] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:24.853282 22384 log.cc:824] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:24.853500 22383 tablet_bootstrap.cc:492] T a5765dcbfb724e179eeed7c1449b2664 P 36d10f923f3a4587b1c5520ab53ad110: No bootstrap required, opened a new log
I20260620 09:25:24.853567 22383 ts_tablet_manager.cc:1397] T a5765dcbfb724e179eeed7c1449b2664 P 36d10f923f3a4587b1c5520ab53ad110: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 09:25:24.854144 22384 tablet_bootstrap.cc:492] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b: No bootstrap required, opened a new log
I20260620 09:25:24.854224 22384 ts_tablet_manager.cc:1397] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:25:24.854869 22382 raft_consensus.cc:348] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d10f923f3a4587b1c5520ab53ad110" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46709 } } peers { permanent_uuid: "60b9f8f5c35c450387355efb92b1d089" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 39675 } } peers { permanent_uuid: "54ebc32f4a824c85924f1f01f375118b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34807 } }
I20260620 09:25:24.855021 22382 raft_consensus.cc:374] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:24.855064 22382 raft_consensus.cc:729] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b9f8f5c35c450387355efb92b1d089, State: Initialized, Role: FOLLOWER
I20260620 09:25:24.855168 22382 consensus_queue.cc:260] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d10f923f3a4587b1c5520ab53ad110" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46709 } } peers { permanent_uuid: "60b9f8f5c35c450387355efb92b1d089" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 39675 } } peers { permanent_uuid: "54ebc32f4a824c85924f1f01f375118b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34807 } }
I20260620 09:25:24.855435 22382 ts_tablet_manager.cc:1428] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:25:24.855453 22368 heartbeater.cc:503] Master 127.4.59.62:32813 was elected leader, sending a full tablet report...
I20260620 09:25:24.856206 22383 raft_consensus.cc:348] T a5765dcbfb724e179eeed7c1449b2664 P 36d10f923f3a4587b1c5520ab53ad110 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d10f923f3a4587b1c5520ab53ad110" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46709 } } peers { permanent_uuid: "60b9f8f5c35c450387355efb92b1d089" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 39675 } } peers { permanent_uuid: "54ebc32f4a824c85924f1f01f375118b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34807 } }
I20260620 09:25:24.856357 22383 raft_consensus.cc:374] T a5765dcbfb724e179eeed7c1449b2664 P 36d10f923f3a4587b1c5520ab53ad110 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:24.856395 22383 raft_consensus.cc:729] T a5765dcbfb724e179eeed7c1449b2664 P 36d10f923f3a4587b1c5520ab53ad110 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d10f923f3a4587b1c5520ab53ad110, State: Initialized, Role: FOLLOWER
I20260620 09:25:24.856495 22383 consensus_queue.cc:260] T a5765dcbfb724e179eeed7c1449b2664 P 36d10f923f3a4587b1c5520ab53ad110 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d10f923f3a4587b1c5520ab53ad110" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46709 } } peers { permanent_uuid: "60b9f8f5c35c450387355efb92b1d089" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 39675 } } peers { permanent_uuid: "54ebc32f4a824c85924f1f01f375118b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34807 } }
I20260620 09:25:24.856736 22383 ts_tablet_manager.cc:1428] T a5765dcbfb724e179eeed7c1449b2664 P 36d10f923f3a4587b1c5520ab53ad110: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:25:24.856779 22106 heartbeater.cc:503] Master 127.4.59.62:32813 was elected leader, sending a full tablet report...
I20260620 09:25:24.856709 22384 raft_consensus.cc:348] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d10f923f3a4587b1c5520ab53ad110" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46709 } } peers { permanent_uuid: "60b9f8f5c35c450387355efb92b1d089" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 39675 } } peers { permanent_uuid: "54ebc32f4a824c85924f1f01f375118b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34807 } }
I20260620 09:25:24.856844 22384 raft_consensus.cc:374] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:24.856884 22384 raft_consensus.cc:729] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ebc32f4a824c85924f1f01f375118b, State: Initialized, Role: FOLLOWER
I20260620 09:25:24.857002 22384 consensus_queue.cc:260] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d10f923f3a4587b1c5520ab53ad110" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46709 } } peers { permanent_uuid: "60b9f8f5c35c450387355efb92b1d089" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 39675 } } peers { permanent_uuid: "54ebc32f4a824c85924f1f01f375118b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34807 } }
I20260620 09:25:24.857275 22384 ts_tablet_manager.cc:1428] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260620 09:25:24.857321 22237 heartbeater.cc:503] Master 127.4.59.62:32813 was elected leader, sending a full tablet report...
I20260620 09:25:24.889550 22390 raft_consensus.cc:482] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:25:24.889690 22390 raft_consensus.cc:504] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d10f923f3a4587b1c5520ab53ad110" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46709 } } peers { permanent_uuid: "60b9f8f5c35c450387355efb92b1d089" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 39675 } } peers { permanent_uuid: "54ebc32f4a824c85924f1f01f375118b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34807 } }
I20260620 09:25:24.889982 22390 leader_election.cc:288] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36d10f923f3a4587b1c5520ab53ad110 (127.4.59.1:46709), 60b9f8f5c35c450387355efb92b1d089 (127.4.59.3:39675)
I20260620 09:25:24.892808 22239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.890108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:46709 (local address 127.4.59.2:42673)
0620 09:25:24.890150 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:24.890163 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:24.890205 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:24.890365 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:24.890366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:24.890367 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:24.890440 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.890441 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.891120 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.891121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.891877 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.891879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.892550 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.892551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.892620 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.892623 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:25:24.892674 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 09:25:24.892715 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":7}
I20260620 09:25:24.892815 22380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.890168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:42673 (local address 127.4.59.1:46709)
0620 09:25:24.890212 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.890215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.890225 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:25:24.890256 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.890256 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.890269 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.890337 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:24.890517 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.891034 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.892068 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.892468 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.892521 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.892529 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.892703 (+ 174us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.892750 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 09:25:24.892889 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.890218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:59939 (local address 127.4.59.3:39675)
0620 09:25:24.890257 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.890258 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.890365 (+ 107us) server_negotiation.cc:407] Connection header received
0620 09:25:24.890415 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.890416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.890427 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.890478 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:24.890634 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.891058 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.892152 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.892491 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.892589 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.892598 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.892780 (+ 182us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.892817 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 09:25:24.892920 22391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.890149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:39675 (local address 127.4.59.2:59939)
0620 09:25:24.890295 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:24.890305 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:24.890356 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:24.890508 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:24.890508 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:24.890510 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:24.890571 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.890573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.891179 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.891179 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.892066 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:24.892068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.892605 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.892606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:24.892690 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.892695 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 09:25:24.892757 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 09:25:24.892790 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260620 09:25:24.893162 22060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5765dcbfb724e179eeed7c1449b2664" candidate_uuid: "54ebc32f4a824c85924f1f01f375118b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d10f923f3a4587b1c5520ab53ad110" is_pre_election: true
I20260620 09:25:24.893240 22322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5765dcbfb724e179eeed7c1449b2664" candidate_uuid: "54ebc32f4a824c85924f1f01f375118b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b9f8f5c35c450387355efb92b1d089" is_pre_election: true
I20260620 09:25:24.893307 22060 raft_consensus.cc:2454] T a5765dcbfb724e179eeed7c1449b2664 P 36d10f923f3a4587b1c5520ab53ad110 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ebc32f4a824c85924f1f01f375118b in term 0.
I20260620 09:25:24.893404 22322 raft_consensus.cc:2454] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ebc32f4a824c85924f1f01f375118b in term 0.
I20260620 09:25:24.893613 22124 leader_election.cc:302] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54ebc32f4a824c85924f1f01f375118b, 60b9f8f5c35c450387355efb92b1d089; no voters:
I20260620 09:25:24.893733 22390 raft_consensus.cc:2781] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:25:24.893784 22390 raft_consensus.cc:482] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:25:24.893818 22390 raft_consensus.cc:3037] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:24.894486 22390 raft_consensus.cc:504] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d10f923f3a4587b1c5520ab53ad110" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46709 } } peers { permanent_uuid: "60b9f8f5c35c450387355efb92b1d089" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 39675 } } peers { permanent_uuid: "54ebc32f4a824c85924f1f01f375118b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34807 } }
I20260620 09:25:24.894697 22390 leader_election.cc:288] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [CANDIDATE]: Term 1 election: Requested vote from peers 36d10f923f3a4587b1c5520ab53ad110 (127.4.59.1:46709), 60b9f8f5c35c450387355efb92b1d089 (127.4.59.3:39675)
I20260620 09:25:24.894778 22322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5765dcbfb724e179eeed7c1449b2664" candidate_uuid: "54ebc32f4a824c85924f1f01f375118b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b9f8f5c35c450387355efb92b1d089"
I20260620 09:25:24.894778 22060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5765dcbfb724e179eeed7c1449b2664" candidate_uuid: "54ebc32f4a824c85924f1f01f375118b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d10f923f3a4587b1c5520ab53ad110"
I20260620 09:25:24.894850 22322 raft_consensus.cc:3037] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:24.894873 22060 raft_consensus.cc:3037] T a5765dcbfb724e179eeed7c1449b2664 P 36d10f923f3a4587b1c5520ab53ad110 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:24.895516 22322 raft_consensus.cc:2454] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ebc32f4a824c85924f1f01f375118b in term 1.
I20260620 09:25:24.895681 22060 raft_consensus.cc:2454] T a5765dcbfb724e179eeed7c1449b2664 P 36d10f923f3a4587b1c5520ab53ad110 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ebc32f4a824c85924f1f01f375118b in term 1.
I20260620 09:25:24.895685 22124 leader_election.cc:302] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54ebc32f4a824c85924f1f01f375118b, 60b9f8f5c35c450387355efb92b1d089; no voters:
I20260620 09:25:24.895785 22390 raft_consensus.cc:2781] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:24.895924 22390 raft_consensus.cc:686] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [term 1 LEADER]: Becoming Leader. State: Replica: 54ebc32f4a824c85924f1f01f375118b, State: Running, Role: LEADER
I20260620 09:25:24.896023 22390 consensus_queue.cc:237] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d10f923f3a4587b1c5520ab53ad110" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46709 } } peers { permanent_uuid: "60b9f8f5c35c450387355efb92b1d089" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 39675 } } peers { permanent_uuid: "54ebc32f4a824c85924f1f01f375118b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34807 } }
I20260620 09:25:24.896791 21917 catalog_manager.cc:5495] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b reported cstate change: term changed from 0 to 1, leader changed from <none> to 54ebc32f4a824c85924f1f01f375118b (127.4.59.2). New cstate: current_term: 1 leader_uuid: "54ebc32f4a824c85924f1f01f375118b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d10f923f3a4587b1c5520ab53ad110" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60b9f8f5c35c450387355efb92b1d089" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 39675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54ebc32f4a824c85924f1f01f375118b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34807 } health_report { overall_health: HEALTHY } } }
I20260620 09:25:24.905478 22381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.903354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47922 (local address 127.4.59.2:34807)
0620 09:25:24.903410 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:24.903412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:24.903420 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:25:24.903458 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:24.903459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:24.903474 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:24.903536 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:25:24.903637 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.904076 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.904790 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:24.904965 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:24.905013 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:24.905198 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:25:24.905297 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:25:24.905359 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:24.905395 (+ 36us) server_negotiation.cc:299] Negotiation successful
0620 09:25:24.905410 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 09:25:24.907191 22060 raft_consensus.cc:1261] T a5765dcbfb724e179eeed7c1449b2664 P 36d10f923f3a4587b1c5520ab53ad110 [term 1 FOLLOWER]: Refusing update from remote peer 54ebc32f4a824c85924f1f01f375118b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:25:24.907191 22322 raft_consensus.cc:1261] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089 [term 1 FOLLOWER]: Refusing update from remote peer 54ebc32f4a824c85924f1f01f375118b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:25:24.907577 22392 consensus_queue.cc:1035] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d10f923f3a4587b1c5520ab53ad110" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:25:24.907713 22390 consensus_queue.cc:1035] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [LEADER]: Connected to new peer: Peer: permanent_uuid: "60b9f8f5c35c450387355efb92b1d089" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 39675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:25:24.910185 22396 mvcc.cc:205] Tried to move back new op lower bound from 7298857062017179648 to 7298857061974851584. Current Snapshot: MvccSnapshot[applied={T|T < 7298857062017179648}]
I20260620 09:25:24.913122 21918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:25:24.913261 21918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:25:24.913990 21918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e357f79099e43c5ba8397e104e1fd23: Sending DeleteTablet for 3 replicas of tablet a5765dcbfb724e179eeed7c1449b2664
I20260620 09:25:24.914338 22040 tablet_service.cc:1514] Processing DeleteTablet for tablet a5765dcbfb724e179eeed7c1449b2664 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:24 UTC) from {username='slave'} at 127.0.0.1:55824
I20260620 09:25:24.914371 22302 tablet_service.cc:1514] Processing DeleteTablet for tablet a5765dcbfb724e179eeed7c1449b2664 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:24 UTC) from {username='slave'} at 127.0.0.1:58622
I20260620 09:25:24.914425 22171 tablet_service.cc:1514] Processing DeleteTablet for tablet a5765dcbfb724e179eeed7c1449b2664 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:24 UTC) from {username='slave'} at 127.0.0.1:47916
I20260620 09:25:24.914700 22407 tablet_replica.cc:330] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b: stopping tablet replica
I20260620 09:25:24.914759 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 21978
I20260620 09:25:24.914801 22407 raft_consensus.cc:2229] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:25:24.914783 22406 tablet_replica.cc:330] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089: stopping tablet replica
I20260620 09:25:24.914914 22407 raft_consensus.cc:2258] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:25:24.915102 22406 raft_consensus.cc:2229] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:25:24.915244 22406 raft_consensus.cc:2258] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:25:24.915959 22406 ts_tablet_manager.cc:1905] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:25:24.916909 22407 ts_tablet_manager.cc:1905] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:25:24.918072 22406 ts_tablet_manager.cc:1918] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:25:24.918215 22406 log.cc:1196] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/a5765dcbfb724e179eeed7c1449b2664
I20260620 09:25:24.918501 22407 ts_tablet_manager.cc:1918] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:25:24.918578 22407 log.cc:1196] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/a5765dcbfb724e179eeed7c1449b2664
I20260620 09:25:24.918648 22406 ts_tablet_manager.cc:1939] T a5765dcbfb724e179eeed7c1449b2664 P 60b9f8f5c35c450387355efb92b1d089: Deleting consensus metadata
I20260620 09:25:24.918839 22407 ts_tablet_manager.cc:1939] T a5765dcbfb724e179eeed7c1449b2664 P 54ebc32f4a824c85924f1f01f375118b: Deleting consensus metadata
I20260620 09:25:24.919132 21902 catalog_manager.cc:4841] TS 60b9f8f5c35c450387355efb92b1d089 (127.4.59.3:39675): tablet a5765dcbfb724e179eeed7c1449b2664 (table test-table [id=eaa5b9f4291947979aeac424439f4993]) successfully deleted
I20260620 09:25:24.919304 21905 catalog_manager.cc:4841] TS 54ebc32f4a824c85924f1f01f375118b (127.4.59.2:34807): tablet a5765dcbfb724e179eeed7c1449b2664 (table test-table [id=eaa5b9f4291947979aeac424439f4993]) successfully deleted
W20260620 09:25:24.921183 21903 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.59.1:46709 (error 108)
I20260620 09:25:24.921658 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 22109
I20260620 09:25:24.923130 22377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:24.922990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:46709 (local address 127.0.0.1:55834)
0620 09:25:24.923036 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:24.923088 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:46709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260620 09:25:24.923489 21903 catalog_manager.cc:4584] TS 36d10f923f3a4587b1c5520ab53ad110 (127.4.59.1:46709): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a5765dcbfb724e179eeed7c1449b2664: Network error: Client connection negotiation failed: client connection to 127.4.59.1:46709: connect: Connection refused (error 111)
I20260620 09:25:24.926088 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 22240
I20260620 09:25:24.932751 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 21887
2026-06-20T09:25:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (695 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:24 dist-test-slave-414w krb5kdc[22411](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:24 dist-test-slave-414w krb5kdc[22411](info): set up 2 sockets
Jun 20 09:25:24 dist-test-slave-414w krb5kdc[22411](info): commencing operation
krb5kdc: starting...
W20260620 09:25:25.973356 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:26 dist-test-slave-414w krb5kdc[22411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:25:26.085454 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:41545
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:44223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:41545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:26.162791 22424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:26.163033 22424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:26.163070 22424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:26.164193 22424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:26.164260 22424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:26.164286 22424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:26.164319 22424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:26.164343 22424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:26.165794 22424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:44223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:41545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:41545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:26.166065 22424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:26.166289 22424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:26.171072 22429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:26.171075 22432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:26.171077 22430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:26.171412 22424 server_base.cc:1034] running on GCE node
I20260620 09:25:26.171625 22424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:26.171905 22424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:26.173058 22424 hybrid_clock.cc:648] HybridClock initialized: now 1781947526173032 us; error 38 us; skew 500 ppm
Jun 20 09:25:26 dist-test-slave-414w krb5kdc[22411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947526, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:26.175503 22424 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:25:26.175966 22424 webserver.cc:460] Webserver started at http://127.4.59.62:37991/ using document root <none> and password file <none>
I20260620 09:25:26.176194 22424 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:26.176261 22424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:26.176411 22424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:26.177497 22424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "f8f991c8950c46e6b4b1bbaf2b106c99"
format_stamp: "Formatted at 2026-06-20 09:25:26 on dist-test-slave-414w"
server_key: "fa2dca5255d63764a5c20827c1f286ef"
server_key_iv: "cd6cf7b5e9a8f2f4ef2b53a953326f76"
server_key_version: "encryptionkey@0"
I20260620 09:25:26.177848 22424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "f8f991c8950c46e6b4b1bbaf2b106c99"
format_stamp: "Formatted at 2026-06-20 09:25:26 on dist-test-slave-414w"
server_key: "fa2dca5255d63764a5c20827c1f286ef"
server_key_iv: "cd6cf7b5e9a8f2f4ef2b53a953326f76"
server_key_version: "encryptionkey@0"
I20260620 09:25:26.179117 22424 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:25:26.179839 22439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:26.180038 22424 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:26.180142 22424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "f8f991c8950c46e6b4b1bbaf2b106c99"
format_stamp: "Formatted at 2026-06-20 09:25:26 on dist-test-slave-414w"
server_key: "fa2dca5255d63764a5c20827c1f286ef"
server_key_iv: "cd6cf7b5e9a8f2f4ef2b53a953326f76"
server_key_version: "encryptionkey@0"
I20260620 09:25:26.180217 22424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:26.198372 22424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:26.200553 22424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:26.200706 22424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:26.204686 22424 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:41545
I20260620 09:25:26.204723 22491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:41545 every 8 connection(s)
I20260620 09:25:26.205061 22424 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:26.205870 22492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:26.207969 22492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99: Bootstrap starting.
I20260620 09:25:26.208464 22492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:26.208694 22492 log.cc:824] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:26.209362 22492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99: No bootstrap required, opened a new log
I20260620 09:25:26.209582 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 22424
I20260620 09:25:26.209707 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:26.209875 4332 external_mini_cluster.cc:1428] Setting key d007e0787ffc1d4e8fe8220debd8acc5
I20260620 09:25:26.211309 22492 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8f991c8950c46e6b4b1bbaf2b106c99" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 41545 } }
I20260620 09:25:26.211465 22492 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:26.211505 22492 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8f991c8950c46e6b4b1bbaf2b106c99, State: Initialized, Role: FOLLOWER
I20260620 09:25:26.211644 22492 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8f991c8950c46e6b4b1bbaf2b106c99" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 41545 } }
I20260620 09:25:26.211731 22492 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:26.211768 22492 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:26.211818 22492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:26.212834 22492 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8f991c8950c46e6b4b1bbaf2b106c99" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 41545 } }
I20260620 09:25:26.212939 22492 leader_election.cc:302] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8f991c8950c46e6b4b1bbaf2b106c99; no voters:
I20260620 09:25:26.213137 22492 leader_election.cc:288] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:26.213181 22497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:26.213287 22497 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [term 1 LEADER]: Becoming Leader. State: Replica: f8f991c8950c46e6b4b1bbaf2b106c99, State: Running, Role: LEADER
I20260620 09:25:26.213367 22497 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8f991c8950c46e6b4b1bbaf2b106c99" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 41545 } }
I20260620 09:25:26.213444 22492 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:26.213663 22497 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8f991c8950c46e6b4b1bbaf2b106c99. Latest consensus state: current_term: 1 leader_uuid: "f8f991c8950c46e6b4b1bbaf2b106c99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8f991c8950c46e6b4b1bbaf2b106c99" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 41545 } } }
I20260620 09:25:26.213737 22497 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:26.213670 22498 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8f991c8950c46e6b4b1bbaf2b106c99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8f991c8950c46e6b4b1bbaf2b106c99" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 41545 } } }
I20260620 09:25:26.213774 22498 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:26.214111 22501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:26.214612 22501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:26.216179 22501 catalog_manager.cc:1347] Generated new cluster ID: 2e0b5b4d8ff048028f5a9f7a12f25f44
I20260620 09:25:26.216233 22501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 09:25:26 dist-test-slave-414w krb5kdc[22411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:26.220700 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.210360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51584 (local address 127.4.59.62:41545)
0620 09:25:26.210539 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:26.210546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:26.210569 (+ 23us) server_negotiation.cc:407] Connection header received
0620 09:25:26.211300 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:26.211306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:26.211351 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:26.211493 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:26.212285 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.213359 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.214457 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.215061 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.215479 (+ 418us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.217619 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:26.217628 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:26.217629 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:26.217641 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:26.219231 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:26.219610 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:26.219613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:26.219613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:26.219718 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:26.219925 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:26.219926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:26.219927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:26.220139 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:26.220313 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:26.220376 (+ 63us) server_negotiation.cc:299] Negotiation successful
0620 09:25:26.220511 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260620 09:25:26.240638 22501 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:26.241323 22501 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:26.250422 22501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99: Generated new TSK 0
I20260620 09:25:26.250576 22501 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:25:26.273250 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:41545
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:44223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:26.352253 22518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:26.352434 22518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:26.352456 22518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:26.353591 22518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:26.353631 22518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:26.353657 22518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:25:26.355038 22518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:44223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:41545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:26.355213 22518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:26.355376 22518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:26.360119 22526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:26.360157 22524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:26.360306 22518 server_base.cc:1034] running on GCE node
W20260620 09:25:26.360126 22523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:26.360653 22518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:26.360904 22518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:26.362063 22518 hybrid_clock.cc:648] HybridClock initialized: now 1781947526362035 us; error 40 us; skew 500 ppm
Jun 20 09:25:26 dist-test-slave-414w krb5kdc[22411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947526, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:26.364557 22518 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:25:26.364977 22518 webserver.cc:460] Webserver started at http://127.4.59.1:33619/ using document root <none> and password file <none>
I20260620 09:25:26.365234 22518 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:26.365298 22518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:26.365435 22518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:26.366626 22518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "ea8d98d0b8134ca4b411631b1d44ac22"
format_stamp: "Formatted at 2026-06-20 09:25:26 on dist-test-slave-414w"
server_key: "a0c2d338193d1aad8ac50f5094d37bdc"
server_key_iv: "556474f3c8ace358cd1176841318b6e2"
server_key_version: "encryptionkey@0"
I20260620 09:25:26.366962 22518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "ea8d98d0b8134ca4b411631b1d44ac22"
format_stamp: "Formatted at 2026-06-20 09:25:26 on dist-test-slave-414w"
server_key: "a0c2d338193d1aad8ac50f5094d37bdc"
server_key_iv: "556474f3c8ace358cd1176841318b6e2"
server_key_version: "encryptionkey@0"
I20260620 09:25:26.368418 22518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:25:26.369062 22533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:26.369262 22518 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:25:26.369349 22518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "ea8d98d0b8134ca4b411631b1d44ac22"
format_stamp: "Formatted at 2026-06-20 09:25:26 on dist-test-slave-414w"
server_key: "a0c2d338193d1aad8ac50f5094d37bdc"
server_key_iv: "556474f3c8ace358cd1176841318b6e2"
server_key_version: "encryptionkey@0"
I20260620 09:25:26.369419 22518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:26.406589 22518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:26.408941 22518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:26.409081 22518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:26.409359 22518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:26.409689 22518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:26.409746 22518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:26.409786 22518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:26.409813 22518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:26.415693 22518 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:46273
I20260620 09:25:26.415742 22646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:46273 every 8 connection(s)
I20260620 09:25:26.416059 22518 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:25:26.417496 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 22518
I20260620 09:25:26.417601 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:25:26.417788 4332 external_mini_cluster.cc:1428] Setting key 8ae8f91233173087a0ef257abef951f6
Jun 20 09:25:26 dist-test-slave-414w krb5kdc[22411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947526, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:26.426123 22649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.416763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:41545 (local address 127.4.59.1:59203)
0620 09:25:26.416923 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:26.416964 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:26.417506 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:26.417774 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:26.417777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:26.418650 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:26.418888 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:26.418893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.419627 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.419629 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:26.420427 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:26.420429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.420848 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.420849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:26.420980 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.421513 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:26.421522 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:26.423055 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:26.424814 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:26.424820 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:26.424821 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:26.425030 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:26.425279 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:26.425280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:26.425280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:26.425376 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:26.425671 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:26.425675 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:26.425780 (+ 105us) client_negotiation.cc:769] Sending connection context
0620 09:25:26.425855 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 09:25:26.425967 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260620 09:25:26.426121 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.416839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:59203 (local address 127.4.59.62:41545)
0620 09:25:26.416901 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:26.416903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:26.417193 (+ 290us) server_negotiation.cc:407] Connection header received
0620 09:25:26.417621 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:26.417622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:26.417638 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:26.417706 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:26.419015 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.419531 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.420528 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.420739 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.420840 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.423154 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:26.423156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:26.423157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:26.423167 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:26.424734 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:26.425108 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:26.425109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:26.425110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:26.425187 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:26.425445 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:26.425446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:26.425447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:26.425594 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:26.425666 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:26.425911 (+ 245us) server_negotiation.cc:299] Negotiation successful
0620 09:25:26.426012 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
I20260620 09:25:26.426652 22647 heartbeater.cc:347] Connected to a master server at 127.4.59.62:41545
I20260620 09:25:26.426761 22647 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:26.426901 22647 heartbeater.cc:511] Master 127.4.59.62:41545 requested a full tablet report, sending...
I20260620 09:25:26.427347 22456 ts_manager.cc:194] Registered new tserver with Master: ea8d98d0b8134ca4b411631b1d44ac22 (127.4.59.1:46273)
I20260620 09:25:26.427959 22456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:59203
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:25:26.444645 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:41545
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:44223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:26.524240 22652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:26.524426 22652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:26.524448 22652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:26.525524 22652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:26.525556 22652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:26.525583 22652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:25:26.526885 22652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:44223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:41545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:26.527071 22652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:26.527231 22652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:26.531955 22660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:26.531967 22658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:26.531980 22657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:26.531997 22652 server_base.cc:1034] running on GCE node
I20260620 09:25:26.532264 22652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:26.532509 22652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:26.533668 22652 hybrid_clock.cc:648] HybridClock initialized: now 1781947526533659 us; error 38 us; skew 500 ppm
Jun 20 09:25:26 dist-test-slave-414w krb5kdc[22411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947526, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:26.536012 22652 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:25:26.536481 22652 webserver.cc:460] Webserver started at http://127.4.59.2:39757/ using document root <none> and password file <none>
I20260620 09:25:26.536696 22652 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:26.536762 22652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:26.536892 22652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:26.537956 22652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "167bd7dbe3ca4267a0a03246094ad28b"
format_stamp: "Formatted at 2026-06-20 09:25:26 on dist-test-slave-414w"
server_key: "df4ba2e42596325a88999ab247899c30"
server_key_iv: "e8feabc126897d10586aaeace70323ba"
server_key_version: "encryptionkey@0"
I20260620 09:25:26.538312 22652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "167bd7dbe3ca4267a0a03246094ad28b"
format_stamp: "Formatted at 2026-06-20 09:25:26 on dist-test-slave-414w"
server_key: "df4ba2e42596325a88999ab247899c30"
server_key_iv: "e8feabc126897d10586aaeace70323ba"
server_key_version: "encryptionkey@0"
I20260620 09:25:26.539583 22652 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 09:25:26.540339 22667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:26.540550 22652 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:25:26.540644 22652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "167bd7dbe3ca4267a0a03246094ad28b"
format_stamp: "Formatted at 2026-06-20 09:25:26 on dist-test-slave-414w"
server_key: "df4ba2e42596325a88999ab247899c30"
server_key_iv: "e8feabc126897d10586aaeace70323ba"
server_key_version: "encryptionkey@0"
I20260620 09:25:26.540728 22652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:26.550313 22652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:26.552182 22652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:26.552340 22652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:26.552567 22652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:26.552867 22652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:26.552922 22652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:26.552961 22652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:26.552989 22652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:26.558950 22652 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:45907
I20260620 09:25:26.559008 22780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:45907 every 8 connection(s)
I20260620 09:25:26.559299 22652 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
Jun 20 09:25:26 dist-test-slave-414w krb5kdc[22411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947526, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:26.567808 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.559981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:45355 (local address 127.4.59.62:41545)
0620 09:25:26.560028 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:26.560029 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:26.560231 (+ 202us) server_negotiation.cc:407] Connection header received
0620 09:25:26.560747 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:26.560748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:26.560758 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:26.560814 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:26.561710 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.562217 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.563161 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.563342 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.563390 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.565323 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:26.565324 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:26.565325 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:26.565331 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:26.566701 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:26.567035 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:26.567036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:26.567037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:26.567086 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:26.567283 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:26.567284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:26.567284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:26.567377 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:26.567425 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:26.567660 (+ 235us) server_negotiation.cc:299] Negotiation successful
0620 09:25:26.567747 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:25:26.567901 22783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.559976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:41545 (local address 127.4.59.2:45355)
0620 09:25:26.560114 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:26.560147 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:26.560611 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:26.560863 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:26.560866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:26.561435 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:26.561630 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:26.561635 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.562297 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.562297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:26.563085 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:26.563086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.563410 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.563410 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:26.563501 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.563945 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:26.563953 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:26.565217 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:26.566767 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:26.566772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:26.566774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:26.566963 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:26.567142 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:26.567142 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:26.567142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:26.567229 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:26.567428 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:26.567431 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:26.567533 (+ 102us) client_negotiation.cc:769] Sending connection context
0620 09:25:26.567615 (+ 82us) client_negotiation.cc:240] Negotiation successful
0620 09:25:26.567723 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":89,"threads_started":1}
I20260620 09:25:26.568347 22781 heartbeater.cc:347] Connected to a master server at 127.4.59.62:41545
I20260620 09:25:26.568424 22781 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:26.568547 22781 heartbeater.cc:511] Master 127.4.59.62:41545 requested a full tablet report, sending...
I20260620 09:25:26.568851 22456 ts_manager.cc:194] Registered new tserver with Master: 167bd7dbe3ca4267a0a03246094ad28b (127.4.59.2:45907)
I20260620 09:25:26.569244 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 22652
I20260620 09:25:26.569363 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:25:26.569357 22456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:45355
I20260620 09:25:26.569574 4332 external_mini_cluster.cc:1428] Setting key f56188ce0fbc1870a2b3b0986da3b61a
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:25:26.593910 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:41545
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:44223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:26.673067 22786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:26.673352 22786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:26.673388 22786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:26.674531 22786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:26.674595 22786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:26.674640 22786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:25:26.676066 22786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:44223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:41545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:26.676319 22786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:26.676535 22786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:26.681571 22792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:26.681666 22791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:26.681571 22794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:26.681772 22786 server_base.cc:1034] running on GCE node
I20260620 09:25:26.681942 22786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:26.682135 22786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:26.683264 22786 hybrid_clock.cc:648] HybridClock initialized: now 1781947526683246 us; error 36 us; skew 500 ppm
Jun 20 09:25:26 dist-test-slave-414w krb5kdc[22411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947526, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:26.685676 22786 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:25:26.686105 22786 webserver.cc:460] Webserver started at http://127.4.59.3:36605/ using document root <none> and password file <none>
I20260620 09:25:26.686282 22786 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:26.686321 22786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:26.686403 22786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:26.687343 22786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "91aba06ce1ca45e781502e1a22398a4c"
format_stamp: "Formatted at 2026-06-20 09:25:26 on dist-test-slave-414w"
server_key: "602d95c01cb37f743822055dae93b29b"
server_key_iv: "ac60af81521cbfa85292d0393e114c53"
server_key_version: "encryptionkey@0"
I20260620 09:25:26.687630 22786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "91aba06ce1ca45e781502e1a22398a4c"
format_stamp: "Formatted at 2026-06-20 09:25:26 on dist-test-slave-414w"
server_key: "602d95c01cb37f743822055dae93b29b"
server_key_iv: "ac60af81521cbfa85292d0393e114c53"
server_key_version: "encryptionkey@0"
I20260620 09:25:26.688874 22786 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 09:25:26.689646 22801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:26.689816 22786 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:26.689898 22786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "91aba06ce1ca45e781502e1a22398a4c"
format_stamp: "Formatted at 2026-06-20 09:25:26 on dist-test-slave-414w"
server_key: "602d95c01cb37f743822055dae93b29b"
server_key_iv: "ac60af81521cbfa85292d0393e114c53"
server_key_version: "encryptionkey@0"
I20260620 09:25:26.689952 22786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:26.709587 22786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:26.711400 22786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:26.711504 22786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:26.711689 22786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:26.711923 22786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:26.711956 22786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:26.711978 22786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:26.711992 22786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:26.717926 22786 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:43705
I20260620 09:25:26.718005 22914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:43705 every 8 connection(s)
I20260620 09:25:26.718305 22786 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
Jun 20 09:25:26 dist-test-slave-414w krb5kdc[22411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947526, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:26.726760 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.718966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:57547 (local address 127.4.59.62:41545)
0620 09:25:26.719044 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:26.719046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:26.719233 (+ 187us) server_negotiation.cc:407] Connection header received
0620 09:25:26.719680 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:26.719681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:26.719691 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:26.719742 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:26.720635 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.721135 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.722110 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.722285 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.722336 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.724271 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:26.724272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:26.724272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:26.724278 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:26.725686 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:26.726018 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:26.726019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:26.726019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:26.726069 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:26.726270 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:26.726271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:26.726271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:26.726365 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:26.726413 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:26.726643 (+ 230us) server_negotiation.cc:299] Negotiation successful
0620 09:25:26.726700 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 09:25:26.726897 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.718960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:41545 (local address 127.4.59.3:57547)
0620 09:25:26.719114 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:26.719147 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:26.719600 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:26.719791 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:26.719794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:26.720332 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:26.720512 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:26.720517 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.721260 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.721261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:26.722038 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:26.722040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.722354 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.722355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:26.722448 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.722851 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:26.722858 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:26.724186 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:26.725751 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:26.725756 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:26.725757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:26.725952 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:26.726126 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:26.726127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:26.726127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:26.726217 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:26.726418 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:26.726421 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:26.726532 (+ 111us) client_negotiation.cc:769] Sending connection context
0620 09:25:26.726610 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 09:25:26.726717 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260620 09:25:26.727355 22915 heartbeater.cc:347] Connected to a master server at 127.4.59.62:41545
I20260620 09:25:26.727468 22915 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:26.727653 22915 heartbeater.cc:511] Master 127.4.59.62:41545 requested a full tablet report, sending...
I20260620 09:25:26.728024 22456 ts_manager.cc:194] Registered new tserver with Master: 91aba06ce1ca45e781502e1a22398a4c (127.4.59.3:43705)
I20260620 09:25:26.728250 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 22786
I20260620 09:25:26.728320 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:25:26.728513 4332 external_mini_cluster.cc:1428] Setting key 4a07bfea3699555e12082f7784b998b1
I20260620 09:25:26.728492 22456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:57547
I20260620 09:25:26.729400 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:25:26.736403 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.730675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51592 (local address 127.4.59.62:41545)
0620 09:25:26.730729 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:26.730731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:26.730840 (+ 109us) server_negotiation.cc:407] Connection header received
0620 09:25:26.730874 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:26.730875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:26.730882 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:26.730933 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:26.731399 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.731829 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.732480 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.732646 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.732700 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.733965 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:26.733966 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:26.733966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:26.733972 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:26.735273 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:26.735653 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:26.735654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:26.735655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:26.735715 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:26.735924 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:26.735925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:26.735925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:26.736056 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:26.736139 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:26.736293 (+ 154us) server_negotiation.cc:299] Negotiation successful
0620 09:25:26.736348 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 09:25:26.737977 22456 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:25:26.739470 22456 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:25:26.745806 22925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.741589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45620 (local address 127.4.59.1:46273)
0620 09:25:26.741782 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:26.741787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:26.741961 (+ 174us) server_negotiation.cc:407] Connection header received
0620 09:25:26.742007 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:26.742009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:26.742023 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:26.742088 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:26.742330 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.743519 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.744813 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.745368 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.745451 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.745512 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:26.745686 (+ 174us) server_negotiation.cc:299] Negotiation successful
0620 09:25:26.745727 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":92,"threads_started":1}
I20260620 09:25:26.745798 22924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.741471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:46273 (local address 127.0.0.1:45620)
0620 09:25:26.741766 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:26.741800 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:26.741874 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:26.742159 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:26.742160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:26.742162 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:26.742242 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:26.742243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.743622 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.743623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:26.744730 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:26.744732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.745465 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.745466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:26.745553 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.745565 (+ 12us) client_negotiation.cc:769] Sending connection context
0620 09:25:26.745668 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 09:25:26.745724 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":5,"spinlock_wait_cycles":768,"thread_start_us":60,"threads_started":1}
I20260620 09:25:26.745950 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.741629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49032 (local address 127.4.59.3:43705)
0620 09:25:26.741862 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:26.741866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:26.741876 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:25:26.741956 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:26.741958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:26.741974 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:26.742052 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:26.742227 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.743253 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.744756 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.745458 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.745537 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.745588 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:26.745835 (+ 247us) server_negotiation.cc:299] Negotiation successful
0620 09:25:26.745872 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":59,"threads_started":1}
I20260620 09:25:26.745950 22928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.741778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55404 (local address 127.4.59.2:45907)
0620 09:25:26.741965 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:26.741969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:26.742114 (+ 145us) server_negotiation.cc:407] Connection header received
0620 09:25:26.742157 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:26.742159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:26.742174 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:26.742243 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:26.742585 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.743662 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.744780 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.745472 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.745536 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.745589 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:26.745836 (+ 247us) server_negotiation.cc:299] Negotiation successful
0620 09:25:26.745874 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":76,"threads_started":1}
I20260620 09:25:26.746361 22923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.741471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:43705 (local address 127.0.0.1:49032)
0620 09:25:26.741763 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:26.741799 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:26.741873 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:26.742069 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:26.742072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:26.742074 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:26.742144 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:26.742147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.743366 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.743367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:26.744681 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:26.744684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.745627 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.745628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:26.745692 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.745695 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:25:26.745744 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 09:25:26.745793 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":2,"thread_start_us":68,"threads_started":1}
I20260620 09:25:26.746490 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.741664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:45907 (local address 127.0.0.1:55404)
0620 09:25:26.742002 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:26.742011 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:26.742062 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:26.742315 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:26.742316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:26.742319 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:26.742466 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:26.742471 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.743773 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.743774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:26.744693 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:26.744696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.745625 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.745627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:26.745692 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.745696 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:25:26.745744 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 09:25:26.745794 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":8,"thread_start_us":41,"threads_started":1}
I20260620 09:25:26.747095 22581 tablet_service.cc:1467] Processing CreateTablet for tablet f0a7f3f813584e8c9fc75e1ef575e980 (DEFAULT_TABLE table=test-table [id=0785f23f1a5e4c9a869e9872c250e985]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:26.747087 22849 tablet_service.cc:1467] Processing CreateTablet for tablet f0a7f3f813584e8c9fc75e1ef575e980 (DEFAULT_TABLE table=test-table [id=0785f23f1a5e4c9a869e9872c250e985]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:26.747092 22715 tablet_service.cc:1467] Processing CreateTablet for tablet f0a7f3f813584e8c9fc75e1ef575e980 (DEFAULT_TABLE table=test-table [id=0785f23f1a5e4c9a869e9872c250e985]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:26.747377 22581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0a7f3f813584e8c9fc75e1ef575e980. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:26.747387 22715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0a7f3f813584e8c9fc75e1ef575e980. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:26.747387 22849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0a7f3f813584e8c9fc75e1ef575e980. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:26.750036 22929 tablet_bootstrap.cc:492] T f0a7f3f813584e8c9fc75e1ef575e980 P ea8d98d0b8134ca4b411631b1d44ac22: Bootstrap starting.
I20260620 09:25:26.750671 22929 tablet_bootstrap.cc:654] T f0a7f3f813584e8c9fc75e1ef575e980 P ea8d98d0b8134ca4b411631b1d44ac22: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:26.750702 22930 tablet_bootstrap.cc:492] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b: Bootstrap starting.
I20260620 09:25:26.751034 22931 tablet_bootstrap.cc:492] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c: Bootstrap starting.
I20260620 09:25:26.751380 22930 tablet_bootstrap.cc:654] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:26.751384 22929 log.cc:824] T f0a7f3f813584e8c9fc75e1ef575e980 P ea8d98d0b8134ca4b411631b1d44ac22: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:26.751677 22931 tablet_bootstrap.cc:654] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:26.751921 22930 log.cc:824] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:26.752226 22929 tablet_bootstrap.cc:492] T f0a7f3f813584e8c9fc75e1ef575e980 P ea8d98d0b8134ca4b411631b1d44ac22: No bootstrap required, opened a new log
I20260620 09:25:26.752280 22931 log.cc:824] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:26.752308 22929 ts_tablet_manager.cc:1397] T f0a7f3f813584e8c9fc75e1ef575e980 P ea8d98d0b8134ca4b411631b1d44ac22: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:25:26.752686 22930 tablet_bootstrap.cc:492] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b: No bootstrap required, opened a new log
I20260620 09:25:26.752791 22930 ts_tablet_manager.cc:1397] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:25:26.752981 22931 tablet_bootstrap.cc:492] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c: No bootstrap required, opened a new log
I20260620 09:25:26.753048 22931 ts_tablet_manager.cc:1397] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:25:26.754411 22931 raft_consensus.cc:348] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8d98d0b8134ca4b411631b1d44ac22" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46273 } } peers { permanent_uuid: "91aba06ce1ca45e781502e1a22398a4c" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43705 } } peers { permanent_uuid: "167bd7dbe3ca4267a0a03246094ad28b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45907 } }
I20260620 09:25:26.754549 22931 raft_consensus.cc:374] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:26.754571 22931 raft_consensus.cc:729] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91aba06ce1ca45e781502e1a22398a4c, State: Initialized, Role: FOLLOWER
I20260620 09:25:26.754484 22929 raft_consensus.cc:348] T f0a7f3f813584e8c9fc75e1ef575e980 P ea8d98d0b8134ca4b411631b1d44ac22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8d98d0b8134ca4b411631b1d44ac22" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46273 } } peers { permanent_uuid: "91aba06ce1ca45e781502e1a22398a4c" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43705 } } peers { permanent_uuid: "167bd7dbe3ca4267a0a03246094ad28b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45907 } }
I20260620 09:25:26.754645 22929 raft_consensus.cc:374] T f0a7f3f813584e8c9fc75e1ef575e980 P ea8d98d0b8134ca4b411631b1d44ac22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:26.754688 22929 raft_consensus.cc:729] T f0a7f3f813584e8c9fc75e1ef575e980 P ea8d98d0b8134ca4b411631b1d44ac22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea8d98d0b8134ca4b411631b1d44ac22, State: Initialized, Role: FOLLOWER
I20260620 09:25:26.754688 22931 consensus_queue.cc:260] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8d98d0b8134ca4b411631b1d44ac22" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46273 } } peers { permanent_uuid: "91aba06ce1ca45e781502e1a22398a4c" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43705 } } peers { permanent_uuid: "167bd7dbe3ca4267a0a03246094ad28b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45907 } }
I20260620 09:25:26.754859 22929 consensus_queue.cc:260] T f0a7f3f813584e8c9fc75e1ef575e980 P ea8d98d0b8134ca4b411631b1d44ac22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8d98d0b8134ca4b411631b1d44ac22" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46273 } } peers { permanent_uuid: "91aba06ce1ca45e781502e1a22398a4c" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43705 } } peers { permanent_uuid: "167bd7dbe3ca4267a0a03246094ad28b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45907 } }
I20260620 09:25:26.754967 22931 ts_tablet_manager.cc:1428] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 09:25:26.755086 22915 heartbeater.cc:503] Master 127.4.59.62:41545 was elected leader, sending a full tablet report...
I20260620 09:25:26.755128 22929 ts_tablet_manager.cc:1428] T f0a7f3f813584e8c9fc75e1ef575e980 P ea8d98d0b8134ca4b411631b1d44ac22: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:25:26.755100 22930 raft_consensus.cc:348] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8d98d0b8134ca4b411631b1d44ac22" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46273 } } peers { permanent_uuid: "91aba06ce1ca45e781502e1a22398a4c" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43705 } } peers { permanent_uuid: "167bd7dbe3ca4267a0a03246094ad28b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45907 } }
I20260620 09:25:26.755246 22647 heartbeater.cc:503] Master 127.4.59.62:41545 was elected leader, sending a full tablet report...
I20260620 09:25:26.755263 22930 raft_consensus.cc:374] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:26.755301 22930 raft_consensus.cc:729] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 167bd7dbe3ca4267a0a03246094ad28b, State: Initialized, Role: FOLLOWER
I20260620 09:25:26.755417 22930 consensus_queue.cc:260] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8d98d0b8134ca4b411631b1d44ac22" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46273 } } peers { permanent_uuid: "91aba06ce1ca45e781502e1a22398a4c" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43705 } } peers { permanent_uuid: "167bd7dbe3ca4267a0a03246094ad28b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45907 } }
I20260620 09:25:26.755690 22930 ts_tablet_manager.cc:1428] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 09:25:26.755716 22781 heartbeater.cc:503] Master 127.4.59.62:41545 was elected leader, sending a full tablet report...
W20260620 09:25:26.809885 22782 tablet.cc:2369] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 09:25:26.916824 22648 tablet.cc:2369] T f0a7f3f813584e8c9fc75e1ef575e980 P ea8d98d0b8134ca4b411631b1d44ac22: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:25:26.956532 22935 raft_consensus.cc:482] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:25:26.956650 22935 raft_consensus.cc:504] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8d98d0b8134ca4b411631b1d44ac22" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46273 } } peers { permanent_uuid: "91aba06ce1ca45e781502e1a22398a4c" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43705 } } peers { permanent_uuid: "167bd7dbe3ca4267a0a03246094ad28b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45907 } }
I20260620 09:25:26.956982 22935 leader_election.cc:288] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea8d98d0b8134ca4b411631b1d44ac22 (127.4.59.1:46273), 167bd7dbe3ca4267a0a03246094ad28b (127.4.59.2:45907)
I20260620 09:25:26.959977 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.957226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:46273 (local address 127.4.59.3:37779)
0620 09:25:26.957390 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:26.957402 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:26.957461 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:26.957651 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:26.957652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:26.957655 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:26.957738 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:26.957740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.958410 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.958411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:26.959164 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:26.959166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.959708 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.959709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:26.959794 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.959798 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:25:26.959849 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 09:25:26.959886 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":10}
I20260620 09:25:26.959983 22925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.957384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:37779 (local address 127.4.59.1:46273)
0620 09:25:26.957453 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:26.957455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:26.957466 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:25:26.957571 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:26.957572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:26.957585 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:26.957637 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:26.957808 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.958332 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.959254 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.959634 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.959686 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.959695 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:26.959865 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 09:25:26.959909 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 09:25:26.960076 22928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.957589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:42209 (local address 127.4.59.2:45907)
0620 09:25:26.957628 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:26.957629 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:26.957637 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:25:26.957664 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:26.957665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:26.957675 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:26.957734 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:26.957926 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.958387 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.959314 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.959699 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.959767 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.959778 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:26.959957 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 09:25:26.960003 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:25:26.960285 22938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.957315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:45907 (local address 127.4.59.3:42209)
0620 09:25:26.957521 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:26.957531 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:26.957580 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:26.957750 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:26.957750 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:26.957752 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:26.957836 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:26.957838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.958476 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.958477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:26.959194 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:26.959195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.959793 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.959794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:26.959879 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.959883 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:25:26.959938 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 09:25:26.959974 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":105,"threads_started":1}
I20260620 09:25:26.960356 22601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0a7f3f813584e8c9fc75e1ef575e980" candidate_uuid: "91aba06ce1ca45e781502e1a22398a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea8d98d0b8134ca4b411631b1d44ac22" is_pre_election: true
I20260620 09:25:26.960500 22601 raft_consensus.cc:2454] T f0a7f3f813584e8c9fc75e1ef575e980 P ea8d98d0b8134ca4b411631b1d44ac22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91aba06ce1ca45e781502e1a22398a4c in term 0.
I20260620 09:25:26.960705 22803 leader_election.cc:302] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91aba06ce1ca45e781502e1a22398a4c, ea8d98d0b8134ca4b411631b1d44ac22; no voters:
I20260620 09:25:26.960745 22735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0a7f3f813584e8c9fc75e1ef575e980" candidate_uuid: "91aba06ce1ca45e781502e1a22398a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "167bd7dbe3ca4267a0a03246094ad28b" is_pre_election: true
I20260620 09:25:26.960860 22935 raft_consensus.cc:2781] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:25:26.960901 22735 raft_consensus.cc:2454] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91aba06ce1ca45e781502e1a22398a4c in term 0.
I20260620 09:25:26.960915 22935 raft_consensus.cc:482] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:25:26.960947 22935 raft_consensus.cc:3037] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:26.961701 22935 raft_consensus.cc:504] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8d98d0b8134ca4b411631b1d44ac22" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46273 } } peers { permanent_uuid: "91aba06ce1ca45e781502e1a22398a4c" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43705 } } peers { permanent_uuid: "167bd7dbe3ca4267a0a03246094ad28b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45907 } }
I20260620 09:25:26.961839 22935 leader_election.cc:288] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [CANDIDATE]: Term 1 election: Requested vote from peers ea8d98d0b8134ca4b411631b1d44ac22 (127.4.59.1:46273), 167bd7dbe3ca4267a0a03246094ad28b (127.4.59.2:45907)
I20260620 09:25:26.962007 22735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0a7f3f813584e8c9fc75e1ef575e980" candidate_uuid: "91aba06ce1ca45e781502e1a22398a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "167bd7dbe3ca4267a0a03246094ad28b"
I20260620 09:25:26.962028 22601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0a7f3f813584e8c9fc75e1ef575e980" candidate_uuid: "91aba06ce1ca45e781502e1a22398a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea8d98d0b8134ca4b411631b1d44ac22"
I20260620 09:25:26.962085 22735 raft_consensus.cc:3037] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:26.962110 22601 raft_consensus.cc:3037] T f0a7f3f813584e8c9fc75e1ef575e980 P ea8d98d0b8134ca4b411631b1d44ac22 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:26.962771 22601 raft_consensus.cc:2454] T f0a7f3f813584e8c9fc75e1ef575e980 P ea8d98d0b8134ca4b411631b1d44ac22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91aba06ce1ca45e781502e1a22398a4c in term 1.
I20260620 09:25:26.962796 22735 raft_consensus.cc:2454] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91aba06ce1ca45e781502e1a22398a4c in term 1.
I20260620 09:25:26.962975 22803 leader_election.cc:302] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91aba06ce1ca45e781502e1a22398a4c, ea8d98d0b8134ca4b411631b1d44ac22; no voters:
I20260620 09:25:26.963063 22935 raft_consensus.cc:2781] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:26.963188 22935 raft_consensus.cc:686] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [term 1 LEADER]: Becoming Leader. State: Replica: 91aba06ce1ca45e781502e1a22398a4c, State: Running, Role: LEADER
I20260620 09:25:26.963279 22935 consensus_queue.cc:237] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8d98d0b8134ca4b411631b1d44ac22" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46273 } } peers { permanent_uuid: "91aba06ce1ca45e781502e1a22398a4c" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43705 } } peers { permanent_uuid: "167bd7dbe3ca4267a0a03246094ad28b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45907 } }
I20260620 09:25:26.964128 22456 catalog_manager.cc:5495] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 91aba06ce1ca45e781502e1a22398a4c (127.4.59.3). New cstate: current_term: 1 leader_uuid: "91aba06ce1ca45e781502e1a22398a4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8d98d0b8134ca4b411631b1d44ac22" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91aba06ce1ca45e781502e1a22398a4c" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "167bd7dbe3ca4267a0a03246094ad28b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45907 } health_report { overall_health: UNKNOWN } } }
W20260620 09:25:26.968806 22916 tablet.cc:2369] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:25:26.971385 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.969179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49038 (local address 127.4.59.3:43705)
0620 09:25:26.969367 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:26.969369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:26.969376 (+ 7us) server_negotiation.cc:407] Connection header received
0620 09:25:26.969400 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:26.969401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:26.969422 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:26.969479 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:25:26.969587 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.970014 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.970732 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:26.970925 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:26.970992 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:26.971110 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:25:26.971190 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:25:26.971256 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:26.971314 (+ 58us) server_negotiation.cc:299] Negotiation successful
0620 09:25:26.971323 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260620 09:25:26.973073 22601 raft_consensus.cc:1261] T f0a7f3f813584e8c9fc75e1ef575e980 P ea8d98d0b8134ca4b411631b1d44ac22 [term 1 FOLLOWER]: Refusing update from remote peer 91aba06ce1ca45e781502e1a22398a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:25:26.973073 22735 raft_consensus.cc:1261] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b [term 1 FOLLOWER]: Refusing update from remote peer 91aba06ce1ca45e781502e1a22398a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:25:26.973451 22939 consensus_queue.cc:1035] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea8d98d0b8134ca4b411631b1d44ac22" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:25:26.973564 22935 consensus_queue.cc:1035] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "167bd7dbe3ca4267a0a03246094ad28b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:25:26.975559 22942 mvcc.cc:205] Tried to move back new op lower bound from 7298857070479060992 to 7298857070442360832. Current Snapshot: MvccSnapshot[applied={T|T < 7298857070479060992}]
I20260620 09:25:26.976142 22944 mvcc.cc:205] Tried to move back new op lower bound from 7298857070479060992 to 7298857070442360832. Current Snapshot: MvccSnapshot[applied={T|T < 7298857070479060992}]
I20260620 09:25:26.978385 22455 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:25:26.978478 22455 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:25:26.979234 22455 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8f991c8950c46e6b4b1bbaf2b106c99: Sending DeleteTablet for 3 replicas of tablet f0a7f3f813584e8c9fc75e1ef575e980
I20260620 09:25:26.979609 22715 tablet_service.cc:1514] Processing DeleteTablet for tablet f0a7f3f813584e8c9fc75e1ef575e980 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:26 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:55404
I20260620 09:25:26.979666 22581 tablet_service.cc:1514] Processing DeleteTablet for tablet f0a7f3f813584e8c9fc75e1ef575e980 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:26 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:45620
I20260620 09:25:26.979743 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 22518
I20260620 09:25:26.980314 22849 tablet_service.cc:1514] Processing DeleteTablet for tablet f0a7f3f813584e8c9fc75e1ef575e980 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:26 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:49032
I20260620 09:25:26.980607 22952 tablet_replica.cc:330] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b: stopping tablet replica
I20260620 09:25:26.980700 22952 raft_consensus.cc:2229] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:25:26.980793 22952 raft_consensus.cc:2258] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:25:26.981277 22952 ts_tablet_manager.cc:1905] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:25:26.984143 22952 ts_tablet_manager.cc:1918] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:25:26.984215 22952 log.cc:1196] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/f0a7f3f813584e8c9fc75e1ef575e980
I20260620 09:25:26.984488 22952 ts_tablet_manager.cc:1939] T f0a7f3f813584e8c9fc75e1ef575e980 P 167bd7dbe3ca4267a0a03246094ad28b: Deleting consensus metadata
I20260620 09:25:26.984915 22441 catalog_manager.cc:4841] TS 167bd7dbe3ca4267a0a03246094ad28b (127.4.59.2:45907): tablet f0a7f3f813584e8c9fc75e1ef575e980 (table test-table [id=0785f23f1a5e4c9a869e9872c250e985]) successfully deleted
W20260620 09:25:26.987448 22441 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.59.1:46273 (error 108)
I20260620 09:25:26.987939 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:26.987796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:46273 (local address 127.0.0.1:45630)
0620 09:25:26.987858 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:26.987906 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:46273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 09:25:26.988006 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 22652
I20260620 09:25:26.988536 22954 tablet_replica.cc:330] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c: stopping tablet replica
I20260620 09:25:26.988638 22954 raft_consensus.cc:2229] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:25:26.988761 22954 raft_consensus.cc:2258] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:25:26.989704 22954 ts_tablet_manager.cc:1905] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:25:26.991655 22954 ts_tablet_manager.cc:1918] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:25:26.991730 22954 log.cc:1196] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/f0a7f3f813584e8c9fc75e1ef575e980
I20260620 09:25:26.992002 22954 ts_tablet_manager.cc:1939] T f0a7f3f813584e8c9fc75e1ef575e980 P 91aba06ce1ca45e781502e1a22398a4c: Deleting consensus metadata
I20260620 09:25:26.992436 22442 catalog_manager.cc:4841] TS 91aba06ce1ca45e781502e1a22398a4c (127.4.59.3:43705): tablet f0a7f3f813584e8c9fc75e1ef575e980 (table test-table [id=0785f23f1a5e4c9a869e9872c250e985]) successfully deleted
W20260620 09:25:26.993299 22441 catalog_manager.cc:4584] TS ea8d98d0b8134ca4b411631b1d44ac22 (127.4.59.1:46273): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f0a7f3f813584e8c9fc75e1ef575e980: Network error: Client connection negotiation failed: client connection to 127.4.59.1:46273: connect: Connection refused (error 111)
I20260620 09:25:26.994202 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 22786
I20260620 09:25:26.999243 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 22424
2026-06-20T09:25:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2066 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-20T09:25:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:27Z Disabled control of system clock
I20260620 09:25:27.026331 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:35399
--webserver_interface=127.4.59.62
--webserver_port=0
--builtin_ntp_servers=127.4.59.20:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:35399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 09:25:27.104199 22959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:27.104393 22959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:27.104415 22959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:27.105489 22959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:27.105522 22959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:27.105535 22959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:27.105547 22959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:27.105561 22959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:27.106786 22959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:35399
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:35399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:27.106967 22959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:27.107175 22959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 09:25:27.111816 22959 server_base.cc:1034] running on GCE node
W20260620 09:25:27.111788 22964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:27.111800 22965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:27.111826 22967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:27.112247 22959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:27.112525 22959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:27.113687 22959 hybrid_clock.cc:648] HybridClock initialized: now 1781947527113648 us; error 49 us; skew 500 ppm
I20260620 09:25:27.114931 22959 webserver.cc:460] Webserver started at http://127.4.59.62:42283/ using document root <none> and password file <none>
I20260620 09:25:27.115173 22959 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:27.115244 22959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:27.115394 22959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:27.116425 22959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "76227e9dd2a843d39753fb754ca49c76"
format_stamp: "Formatted at 2026-06-20 09:25:27 on dist-test-slave-414w"
server_key: "36e0c841bc4f049ac398286220863774"
server_key_iv: "dcd92746256c2da9f6281adc5faebf2a"
server_key_version: "encryptionkey@0"
I20260620 09:25:27.116770 22959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "76227e9dd2a843d39753fb754ca49c76"
format_stamp: "Formatted at 2026-06-20 09:25:27 on dist-test-slave-414w"
server_key: "36e0c841bc4f049ac398286220863774"
server_key_iv: "dcd92746256c2da9f6281adc5faebf2a"
server_key_version: "encryptionkey@0"
I20260620 09:25:27.118084 22959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:25:27.118762 22973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:27.118955 22959 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:27.119033 22959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "76227e9dd2a843d39753fb754ca49c76"
format_stamp: "Formatted at 2026-06-20 09:25:27 on dist-test-slave-414w"
server_key: "36e0c841bc4f049ac398286220863774"
server_key_iv: "dcd92746256c2da9f6281adc5faebf2a"
server_key_version: "encryptionkey@0"
I20260620 09:25:27.119086 22959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:27.141381 22959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:27.141615 22959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:27.141714 22959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:27.145326 22959 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:35399
I20260620 09:25:27.145375 23025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:35399 every 8 connection(s)
I20260620 09:25:27.145689 22959 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:27.146574 23026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:27.148691 23026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76: Bootstrap starting.
I20260620 09:25:27.149267 23026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:27.149549 23026 log.cc:824] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:27.150118 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 22959
I20260620 09:25:27.150228 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:27.150287 23026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76: No bootstrap required, opened a new log
I20260620 09:25:27.150414 4332 external_mini_cluster.cc:1428] Setting key 1ccae26b96652eb0e9b202480aac1d5e
I20260620 09:25:27.152092 23026 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76227e9dd2a843d39753fb754ca49c76" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35399 } }
I20260620 09:25:27.152220 23026 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:27.152240 23026 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76227e9dd2a843d39753fb754ca49c76, State: Initialized, Role: FOLLOWER
I20260620 09:25:27.152375 23026 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76227e9dd2a843d39753fb754ca49c76" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35399 } }
I20260620 09:25:27.152451 23026 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:27.152477 23026 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:27.152530 23026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:27.153105 23026 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76227e9dd2a843d39753fb754ca49c76" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35399 } }
I20260620 09:25:27.153270 23026 leader_election.cc:302] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76227e9dd2a843d39753fb754ca49c76; no voters:
I20260620 09:25:27.153436 23026 leader_election.cc:288] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:27.153481 23031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:27.153601 23031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [term 1 LEADER]: Becoming Leader. State: Replica: 76227e9dd2a843d39753fb754ca49c76, State: Running, Role: LEADER
I20260620 09:25:27.153682 23031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76227e9dd2a843d39753fb754ca49c76" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35399 } }
I20260620 09:25:27.153689 23026 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:27.154937 23032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76227e9dd2a843d39753fb754ca49c76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76227e9dd2a843d39753fb754ca49c76" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35399 } } }
I20260620 09:25:27.155037 23032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:27.155151 23033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76227e9dd2a843d39753fb754ca49c76. Latest consensus state: current_term: 1 leader_uuid: "76227e9dd2a843d39753fb754ca49c76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76227e9dd2a843d39753fb754ca49c76" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 35399 } } }
I20260620 09:25:27.155452 23033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76 [sys.catalog]: This master's current role is: LEADER
W20260620 09:25:27.155646 23045 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:25:27.155701 23045 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:25:27.155756 23046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:27.155896 23046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:27.157467 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.150991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60078 (local address 127.4.59.62:35399)
0620 09:25:27.151198 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:27.151205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:27.151231 (+ 26us) server_negotiation.cc:407] Connection header received
0620 09:25:27.152086 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:27.152092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:27.152145 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:27.152284 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:27.152582 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.154128 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.155140 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.155749 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.155838 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.156770 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:27.156774 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:27.156774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:27.156972 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:27.157011 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:27.157026 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:27.157091 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:27.157201 (+ 110us) server_negotiation.cc:299] Negotiation successful
0620 09:25:27.157285 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
I20260620 09:25:27.158187 23046 catalog_manager.cc:1347] Generated new cluster ID: 4a32309ac4834b5dba763e84dc75d787
I20260620 09:25:27.158243 23046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:27.168288 23046 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:27.168954 23046 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:27.173956 23046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76: Generated new TSK 0
I20260620 09:25:27.174108 23046 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:27.177276 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35399
--builtin_ntp_servers=127.4.59.20:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 09:25:27.257699 23050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:27.257920 23050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:27.257992 23050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:27.259207 23050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:27.259284 23050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:27.259331 23050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:25:27.260815 23050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:27.261066 23050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:27.261369 23050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:27.266055 23058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:27.266047 23056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:27.266047 23055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:27.266248 23050 server_base.cc:1034] running on GCE node
I20260620 09:25:27.266434 23050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:27.266681 23050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:27.267856 23050 hybrid_clock.cc:648] HybridClock initialized: now 1781947527267821 us; error 50 us; skew 500 ppm
I20260620 09:25:27.269198 23050 webserver.cc:460] Webserver started at http://127.4.59.1:36599/ using document root <none> and password file <none>
I20260620 09:25:27.269448 23050 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:27.269501 23050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:27.269624 23050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:27.270882 23050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "3681775a4f83438ab741263e5996b97c"
format_stamp: "Formatted at 2026-06-20 09:25:27 on dist-test-slave-414w"
server_key: "974a897544431f8c8f7f8af7a1a2251b"
server_key_iv: "4ea06e019f2d776d0309d083c2132510"
server_key_version: "encryptionkey@0"
I20260620 09:25:27.271258 23050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "3681775a4f83438ab741263e5996b97c"
format_stamp: "Formatted at 2026-06-20 09:25:27 on dist-test-slave-414w"
server_key: "974a897544431f8c8f7f8af7a1a2251b"
server_key_iv: "4ea06e019f2d776d0309d083c2132510"
server_key_version: "encryptionkey@0"
I20260620 09:25:27.272753 23050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:25:27.273605 23064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:27.273803 23050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 09:25:27.273882 23050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "3681775a4f83438ab741263e5996b97c"
format_stamp: "Formatted at 2026-06-20 09:25:27 on dist-test-slave-414w"
server_key: "974a897544431f8c8f7f8af7a1a2251b"
server_key_iv: "4ea06e019f2d776d0309d083c2132510"
server_key_version: "encryptionkey@0"
I20260620 09:25:27.273948 23050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:27.291620 23050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:27.291836 23050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:27.291929 23050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:27.292098 23050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:27.292405 23050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:27.292462 23050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:27.292501 23050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:27.292531 23050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:27.298208 23050 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:46159
I20260620 09:25:27.298254 23177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:46159 every 8 connection(s)
I20260620 09:25:27.298571 23050 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:25:27.301945 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 23050
I20260620 09:25:27.302050 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:25:27.302255 4332 external_mini_cluster.cc:1428] Setting key bd60a35f6e6935a6a555a0dd8b880f31
I20260620 09:25:27.303539 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35399
--builtin_ntp_servers=127.4.59.20:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 09:25:27.308197 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.299237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:52349 (local address 127.4.59.62:35399)
0620 09:25:27.299294 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:27.299295 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:27.299505 (+ 210us) server_negotiation.cc:407] Connection header received
0620 09:25:27.299911 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:27.299912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:27.299922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:27.299973 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:27.300565 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.301050 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.302204 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.302443 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.302499 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.307938 (+ 5439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:27.307939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:27.307939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:27.307945 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:27.307958 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:27.307961 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:27.308009 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:27.308119 (+ 110us) server_negotiation.cc:299] Negotiation successful
0620 09:25:27.308142 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 09:25:27.308457 23180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.299219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35399 (local address 127.4.59.1:52349)
0620 09:25:27.299381 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:27.299410 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:27.299834 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:27.300023 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:27.300026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:27.300033 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:27.300489 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:27.300493 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.301126 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.301127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:27.302116 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:27.302119 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.302526 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.302527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:27.302696 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.307767 (+ 5071us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:25:27.307768 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:27.307808 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:25:27.307840 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:27.308030 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:27.308036 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:27.308039 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:25:27.308117 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 09:25:27.308232 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260620 09:25:27.308966 23178 heartbeater.cc:347] Connected to a master server at 127.4.59.62:35399
I20260620 09:25:27.309064 23178 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:27.309293 23178 heartbeater.cc:511] Master 127.4.59.62:35399 requested a full tablet report, sending...
I20260620 09:25:27.309789 22990 ts_manager.cc:194] Registered new tserver with Master: 3681775a4f83438ab741263e5996b97c (127.4.59.1:46159)
I20260620 09:25:27.310374 22990 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.59.1:52349
W20260620 09:25:27.383672 23181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:27.383852 23181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:27.383872 23181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:27.385017 23181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:27.385056 23181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:27.385106 23181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:25:27.386449 23181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:27.386669 23181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:27.386934 23181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:27.391621 23187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:27.391629 23186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:27.391683 23189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:27.391880 23181 server_base.cc:1034] running on GCE node
I20260620 09:25:27.392061 23181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:27.392241 23181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:27.393405 23181 hybrid_clock.cc:648] HybridClock initialized: now 1781947527393384 us; error 39 us; skew 500 ppm
I20260620 09:25:27.394501 23181 webserver.cc:460] Webserver started at http://127.4.59.2:35667/ using document root <none> and password file <none>
I20260620 09:25:27.394727 23181 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:27.394794 23181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:27.394953 23181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:27.395964 23181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "ebc4fe782cec46679c636e1dd67cedf8"
format_stamp: "Formatted at 2026-06-20 09:25:27 on dist-test-slave-414w"
server_key: "b3620a002baffe990e5b7f2f20ad7655"
server_key_iv: "b41d8723a47125da6361f75efa570287"
server_key_version: "encryptionkey@0"
I20260620 09:25:27.396310 23181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "ebc4fe782cec46679c636e1dd67cedf8"
format_stamp: "Formatted at 2026-06-20 09:25:27 on dist-test-slave-414w"
server_key: "b3620a002baffe990e5b7f2f20ad7655"
server_key_iv: "b41d8723a47125da6361f75efa570287"
server_key_version: "encryptionkey@0"
I20260620 09:25:27.397609 23181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:25:27.398343 23195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:27.398507 23181 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:25:27.398608 23181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "ebc4fe782cec46679c636e1dd67cedf8"
format_stamp: "Formatted at 2026-06-20 09:25:27 on dist-test-slave-414w"
server_key: "b3620a002baffe990e5b7f2f20ad7655"
server_key_iv: "b41d8723a47125da6361f75efa570287"
server_key_version: "encryptionkey@0"
I20260620 09:25:27.398679 23181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:27.413358 23181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:27.413635 23181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:27.413793 23181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:27.414022 23181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:27.414332 23181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:27.414386 23181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:27.414427 23181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:27.414456 23181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:27.420030 23181 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:33207
I20260620 09:25:27.420100 23308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:33207 every 8 connection(s)
I20260620 09:25:27.420411 23181 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:25:27.425302 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.421133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:53827 (local address 127.4.59.62:35399)
0620 09:25:27.421231 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:27.421233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:27.421399 (+ 166us) server_negotiation.cc:407] Connection header received
0620 09:25:27.421866 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:27.421867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:27.421876 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:27.421926 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:27.422633 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.423163 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.424044 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.424218 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.424265 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.425058 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:27.425059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:27.425060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:27.425063 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:27.425074 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:27.425077 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:27.425116 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:27.425225 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 09:25:27.425245 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 09:25:27.425491 23311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.421091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35399 (local address 127.4.59.2:53827)
0620 09:25:27.421291 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:27.421322 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:27.421788 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:27.422021 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:27.422024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:27.422032 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:27.422565 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:27.422569 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.423240 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.423240 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:27.423975 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:27.423977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.424287 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.424288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:27.424393 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.424936 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:25:27.424936 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:27.424963 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:25:27.424983 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:27.425117 (+ 134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:27.425119 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:27.425121 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:25:27.425221 (+ 100us) client_negotiation.cc:240] Negotiation successful
0620 09:25:27.425294 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":112,"threads_started":1}
I20260620 09:25:27.425987 23309 heartbeater.cc:347] Connected to a master server at 127.4.59.62:35399
I20260620 09:25:27.426059 23309 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:27.426241 23309 heartbeater.cc:511] Master 127.4.59.62:35399 requested a full tablet report, sending...
I20260620 09:25:27.426589 22990 ts_manager.cc:194] Registered new tserver with Master: ebc4fe782cec46679c636e1dd67cedf8 (127.4.59.2:33207)
I20260620 09:25:27.426939 22990 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.59.2:53827
I20260620 09:25:27.427956 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 23181
I20260620 09:25:27.428042 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:25:27.428220 4332 external_mini_cluster.cc:1428] Setting key 9948202a0185d4b3247155050a875c7f
I20260620 09:25:27.429016 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35399
--builtin_ntp_servers=127.4.59.20:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 09:25:27.508831 23312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:27.509003 23312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:27.509023 23312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:27.510139 23312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:27.510180 23312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:27.510205 23312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:25:27.511487 23312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:35399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:27.511703 23312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:27.511960 23312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:27.516685 23317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:27.516711 23318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:27.516695 23320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:27.516716 23312 server_base.cc:1034] running on GCE node
I20260620 09:25:27.517207 23312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:27.517452 23312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:27.518606 23312 hybrid_clock.cc:648] HybridClock initialized: now 1781947527518592 us; error 27 us; skew 500 ppm
I20260620 09:25:27.519755 23312 webserver.cc:460] Webserver started at http://127.4.59.3:41567/ using document root <none> and password file <none>
I20260620 09:25:27.520005 23312 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:27.520073 23312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:27.520223 23312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:27.521365 23312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "f8cf09b32dc94f6081112fc10bff10db"
format_stamp: "Formatted at 2026-06-20 09:25:27 on dist-test-slave-414w"
server_key: "8a201a3ca42ef2e96a93b774f8e51a0a"
server_key_iv: "4364a704ab5b979b6dba73a1e8eede80"
server_key_version: "encryptionkey@0"
I20260620 09:25:27.521708 23312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "f8cf09b32dc94f6081112fc10bff10db"
format_stamp: "Formatted at 2026-06-20 09:25:27 on dist-test-slave-414w"
server_key: "8a201a3ca42ef2e96a93b774f8e51a0a"
server_key_iv: "4364a704ab5b979b6dba73a1e8eede80"
server_key_version: "encryptionkey@0"
I20260620 09:25:27.523064 23312 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:27.523746 23326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:27.523928 23312 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:27.524020 23312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "f8cf09b32dc94f6081112fc10bff10db"
format_stamp: "Formatted at 2026-06-20 09:25:27 on dist-test-slave-414w"
server_key: "8a201a3ca42ef2e96a93b774f8e51a0a"
server_key_iv: "4364a704ab5b979b6dba73a1e8eede80"
server_key_version: "encryptionkey@0"
I20260620 09:25:27.524106 23312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:27.536149 23312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:27.536401 23312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:27.536543 23312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:27.536742 23312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:27.537045 23312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:27.537098 23312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:27.537132 23312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:27.537181 23312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:27.542557 23312 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:45055
I20260620 09:25:27.542594 23439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:45055 every 8 connection(s)
I20260620 09:25:27.542950 23312 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:25:27.543273 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 23312
I20260620 09:25:27.543375 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:25:27.543572 4332 external_mini_cluster.cc:1428] Setting key a00a30168e04d8c340b99d5ed2cf3020
I20260620 09:25:27.547715 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.543720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:48157 (local address 127.4.59.62:35399)
0620 09:25:27.543773 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:27.543775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:27.543989 (+ 214us) server_negotiation.cc:407] Connection header received
0620 09:25:27.544588 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:27.544589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:27.544598 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:27.544641 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:27.545292 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.545785 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.546665 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.546841 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.546889 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.547534 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:27.547535 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:27.547536 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:27.547539 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:27.547551 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:27.547555 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:27.547595 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:27.547641 (+ 46us) server_negotiation.cc:299] Negotiation successful
0620 09:25:27.547662 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 09:25:27.547904 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.543674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:35399 (local address 127.4.59.3:48157)
0620 09:25:27.543864 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:27.543901 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:27.544489 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:27.544686 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:27.544689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:27.544696 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:27.545208 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:27.545213 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.545860 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.545861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:27.546597 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:27.546598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.546908 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.546908 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:27.547015 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.547421 (+ 406us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:25:27.547422 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:27.547446 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:25:27.547465 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:27.547595 (+ 130us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:27.547597 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:27.547599 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:25:27.547641 (+ 42us) client_negotiation.cc:240] Negotiation successful
0620 09:25:27.547710 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":78,"threads_started":1}
I20260620 09:25:27.548385 23440 heartbeater.cc:347] Connected to a master server at 127.4.59.62:35399
I20260620 09:25:27.548480 23440 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:27.548641 23440 heartbeater.cc:511] Master 127.4.59.62:35399 requested a full tablet report, sending...
I20260620 09:25:27.548954 22990 ts_manager.cc:194] Registered new tserver with Master: f8cf09b32dc94f6081112fc10bff10db (127.4.59.3:45055)
I20260620 09:25:27.549343 22990 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.59.3:48157
I20260620 09:25:27.555378 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:25:27.559707 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.556741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60080 (local address 127.4.59.62:35399)
0620 09:25:27.556791 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:27.556792 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:27.556954 (+ 162us) server_negotiation.cc:407] Connection header received
0620 09:25:27.557031 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:27.557032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:27.557046 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:27.557112 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:27.557309 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.557756 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.558523 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.558706 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.558788 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.559424 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:27.559425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:27.559425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:27.559430 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:27.559442 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:27.559446 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:27.559552 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:27.559618 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 09:25:27.559628 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 09:25:27.561077 22990 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:25:27.562642 22990 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:25:27.569159 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.565168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37620 (local address 127.4.59.2:33207)
0620 09:25:27.565547 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:27.565550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:27.565557 (+ 7us) server_negotiation.cc:407] Connection header received
0620 09:25:27.565584 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:27.565586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:27.565704 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:27.565780 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:27.565977 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.566966 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.568025 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.568734 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.568814 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.568903 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:27.569018 (+ 115us) server_negotiation.cc:299] Negotiation successful
0620 09:25:27.569054 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":93,"threads_started":1}
I20260620 09:25:27.569134 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.565175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41672 (local address 127.4.59.3:45055)
0620 09:25:27.565337 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:27.565346 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:27.565354 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:25:27.565384 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:27.565385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:27.565396 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:27.565459 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:27.565641 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.566754 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.568160 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.568830 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.568891 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.568985 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:27.569013 (+ 28us) server_negotiation.cc:299] Negotiation successful
0620 09:25:27.569050 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260620 09:25:27.569166 23451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.565017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:33207 (local address 127.0.0.1:37620)
0620 09:25:27.565340 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:27.565348 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:27.565381 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:27.565845 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:27.565846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:27.565848 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:27.565885 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:27.565890 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.567049 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.567050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:27.567937 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:27.567939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.568817 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.568818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:27.568896 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.568910 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 09:25:27.568991 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 09:25:27.569056 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":3,"thread_start_us":70,"threads_started":1}
I20260620 09:25:27.569166 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.564862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:45055 (local address 127.0.0.1:41672)
0620 09:25:27.565100 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:27.565112 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:27.565183 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:27.565488 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:27.565491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:27.565493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:27.565558 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:27.565560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.566832 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.566833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:27.568077 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:27.568080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.568899 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.568900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:27.568949 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.568953 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:25:27.568994 (+ 41us) client_negotiation.cc:240] Negotiation successful
0620 09:25:27.569056 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":61,"thread_start_us":83,"threads_started":1}
I20260620 09:25:27.569926 23243 tablet_service.cc:1467] Processing CreateTablet for tablet 77b893d9e4a04007b9f3465437530284 (DEFAULT_TABLE table=test-table [id=5f3422dae40f4673bdadffecd573ff29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:27.570287 23243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77b893d9e4a04007b9f3465437530284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:27.570299 23449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.564819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:46159 (local address 127.0.0.1:58580)
0620 09:25:27.565010 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:27.565069 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:27.565136 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:27.565870 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:27.565871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:27.565873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:27.565911 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:27.565913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.566736 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.566737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:27.568077 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:27.568080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.570093 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.570094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:27.570156 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.570160 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:25:27.570193 (+ 33us) client_negotiation.cc:240] Negotiation successful
0620 09:25:27.570232 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":51,"thread_start_us":60,"threads_started":1}
I20260620 09:25:27.570225 23374 tablet_service.cc:1467] Processing CreateTablet for tablet 77b893d9e4a04007b9f3465437530284 (DEFAULT_TABLE table=test-table [id=5f3422dae40f4673bdadffecd573ff29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:27.570521 23374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77b893d9e4a04007b9f3465437530284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:27.570681 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.564906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58580 (local address 127.4.59.1:46159)
0620 09:25:27.565680 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:27.565702 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:27.565712 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:25:27.565749 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:27.565751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:27.565766 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:27.565870 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:27.565976 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.566650 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.569258 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.570001 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.570419 (+ 418us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.570537 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:27.570562 (+ 25us) server_negotiation.cc:299] Negotiation successful
0620 09:25:27.570602 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":99,"threads_started":1}
I20260620 09:25:27.571213 23112 tablet_service.cc:1467] Processing CreateTablet for tablet 77b893d9e4a04007b9f3465437530284 (DEFAULT_TABLE table=test-table [id=5f3422dae40f4673bdadffecd573ff29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:27.571503 23112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77b893d9e4a04007b9f3465437530284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:27.573606 23455 tablet_bootstrap.cc:492] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8: Bootstrap starting.
I20260620 09:25:27.573766 23454 tablet_bootstrap.cc:492] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db: Bootstrap starting.
I20260620 09:25:27.574242 23456 tablet_bootstrap.cc:492] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c: Bootstrap starting.
I20260620 09:25:27.574381 23454 tablet_bootstrap.cc:654] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:27.574520 23455 tablet_bootstrap.cc:654] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:27.574770 23454 log.cc:824] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:27.574899 23455 log.cc:824] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:27.574914 23456 tablet_bootstrap.cc:654] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:27.575218 23456 log.cc:824] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:27.575475 23454 tablet_bootstrap.cc:492] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db: No bootstrap required, opened a new log
I20260620 09:25:27.575528 23454 ts_tablet_manager.cc:1397] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:25:27.575781 23455 tablet_bootstrap.cc:492] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8: No bootstrap required, opened a new log
I20260620 09:25:27.575870 23455 ts_tablet_manager.cc:1397] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:25:27.575943 23456 tablet_bootstrap.cc:492] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c: No bootstrap required, opened a new log
I20260620 09:25:27.576020 23456 ts_tablet_manager.cc:1397] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:25:27.577881 23455 raft_consensus.cc:348] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3681775a4f83438ab741263e5996b97c" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46159 } } peers { permanent_uuid: "f8cf09b32dc94f6081112fc10bff10db" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 45055 } } peers { permanent_uuid: "ebc4fe782cec46679c636e1dd67cedf8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 33207 } }
I20260620 09:25:27.578094 23455 raft_consensus.cc:374] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:27.578145 23455 raft_consensus.cc:729] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebc4fe782cec46679c636e1dd67cedf8, State: Initialized, Role: FOLLOWER
I20260620 09:25:27.578095 23454 raft_consensus.cc:348] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3681775a4f83438ab741263e5996b97c" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46159 } } peers { permanent_uuid: "f8cf09b32dc94f6081112fc10bff10db" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 45055 } } peers { permanent_uuid: "ebc4fe782cec46679c636e1dd67cedf8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 33207 } }
I20260620 09:25:27.578253 23454 raft_consensus.cc:374] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:27.578292 23454 raft_consensus.cc:729] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8cf09b32dc94f6081112fc10bff10db, State: Initialized, Role: FOLLOWER
I20260620 09:25:27.578269 23455 consensus_queue.cc:260] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3681775a4f83438ab741263e5996b97c" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46159 } } peers { permanent_uuid: "f8cf09b32dc94f6081112fc10bff10db" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 45055 } } peers { permanent_uuid: "ebc4fe782cec46679c636e1dd67cedf8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 33207 } }
I20260620 09:25:27.578400 23454 consensus_queue.cc:260] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3681775a4f83438ab741263e5996b97c" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46159 } } peers { permanent_uuid: "f8cf09b32dc94f6081112fc10bff10db" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 45055 } } peers { permanent_uuid: "ebc4fe782cec46679c636e1dd67cedf8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 33207 } }
I20260620 09:25:27.578550 23455 ts_tablet_manager.cc:1428] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 09:25:27.578485 23456 raft_consensus.cc:348] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3681775a4f83438ab741263e5996b97c" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46159 } } peers { permanent_uuid: "f8cf09b32dc94f6081112fc10bff10db" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 45055 } } peers { permanent_uuid: "ebc4fe782cec46679c636e1dd67cedf8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 33207 } }
I20260620 09:25:27.578627 23456 raft_consensus.cc:374] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:27.578648 23454 ts_tablet_manager.cc:1428] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:25:27.578675 23456 raft_consensus.cc:729] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3681775a4f83438ab741263e5996b97c, State: Initialized, Role: FOLLOWER
I20260620 09:25:27.578747 23440 heartbeater.cc:503] Master 127.4.59.62:35399 was elected leader, sending a full tablet report...
I20260620 09:25:27.578787 23309 heartbeater.cc:503] Master 127.4.59.62:35399 was elected leader, sending a full tablet report...
I20260620 09:25:27.579062 23456 consensus_queue.cc:260] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3681775a4f83438ab741263e5996b97c" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46159 } } peers { permanent_uuid: "f8cf09b32dc94f6081112fc10bff10db" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 45055 } } peers { permanent_uuid: "ebc4fe782cec46679c636e1dd67cedf8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 33207 } }
I20260620 09:25:27.579592 23456 ts_tablet_manager.cc:1428] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260620 09:25:27.579707 23178 heartbeater.cc:503] Master 127.4.59.62:35399 was elected leader, sending a full tablet report...
I20260620 09:25:27.650449 23462 raft_consensus.cc:482] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:25:27.650571 23462 raft_consensus.cc:504] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3681775a4f83438ab741263e5996b97c" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46159 } } peers { permanent_uuid: "f8cf09b32dc94f6081112fc10bff10db" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 45055 } } peers { permanent_uuid: "ebc4fe782cec46679c636e1dd67cedf8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 33207 } }
I20260620 09:25:27.650887 23462 leader_election.cc:288] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8cf09b32dc94f6081112fc10bff10db (127.4.59.3:45055), ebc4fe782cec46679c636e1dd67cedf8 (127.4.59.2:33207)
I20260620 09:25:27.654001 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.651203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:54853 (local address 127.4.59.3:45055)
0620 09:25:27.651248 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:27.651250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:27.651259 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:25:27.651290 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:27.651291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:27.651299 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:27.651359 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:27.651536 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.652116 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.653102 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.653613 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.653693 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.653703 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:27.653891 (+ 188us) server_negotiation.cc:299] Negotiation successful
0620 09:25:27.653931 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 09:25:27.654003 23180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.651078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:45055 (local address 127.4.59.1:54853)
0620 09:25:27.651139 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:27.651154 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:27.651208 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:27.651383 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:27.651384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:27.651385 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:27.651463 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:27.651464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.652197 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.652198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:27.653004 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:27.653006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.653725 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.653726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:27.653804 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.653807 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:25:27.653871 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 09:25:27.653931 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":1}
I20260620 09:25:27.654102 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.651185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:51287 (local address 127.4.59.2:33207)
0620 09:25:27.651239 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:27.651241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:27.651421 (+ 180us) server_negotiation.cc:407] Connection header received
0620 09:25:27.651459 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:27.651460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:27.651468 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:27.651541 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:27.651700 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.652275 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.653237 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.653722 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.653811 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.653821 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:27.653999 (+ 178us) server_negotiation.cc:299] Negotiation successful
0620 09:25:27.654035 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 09:25:27.654241 23463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.651078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:33207 (local address 127.4.59.1:51287)
0620 09:25:27.651315 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:27.651324 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:27.651397 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:27.651547 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:27.651548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:27.651549 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:27.651608 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:27.651609 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.652354 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.652355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:27.653095 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:27.653097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.653815 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.653816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:27.653926 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.653930 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:25:27.653991 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 09:25:27.654031 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":67,"thread_start_us":95,"threads_started":1}
I20260620 09:25:27.654456 23394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77b893d9e4a04007b9f3465437530284" candidate_uuid: "3681775a4f83438ab741263e5996b97c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8cf09b32dc94f6081112fc10bff10db" is_pre_election: true
I20260620 09:25:27.654609 23394 raft_consensus.cc:2454] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3681775a4f83438ab741263e5996b97c in term 0.
I20260620 09:25:27.654659 23263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77b893d9e4a04007b9f3465437530284" candidate_uuid: "3681775a4f83438ab741263e5996b97c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebc4fe782cec46679c636e1dd67cedf8" is_pre_election: true
I20260620 09:25:27.654815 23263 raft_consensus.cc:2454] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3681775a4f83438ab741263e5996b97c in term 0.
I20260620 09:25:27.654850 23065 leader_election.cc:302] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3681775a4f83438ab741263e5996b97c, f8cf09b32dc94f6081112fc10bff10db; no voters:
I20260620 09:25:27.654966 23462 raft_consensus.cc:2781] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:25:27.655041 23462 raft_consensus.cc:482] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:25:27.655067 23462 raft_consensus.cc:3037] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:27.655849 23462 raft_consensus.cc:504] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3681775a4f83438ab741263e5996b97c" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46159 } } peers { permanent_uuid: "f8cf09b32dc94f6081112fc10bff10db" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 45055 } } peers { permanent_uuid: "ebc4fe782cec46679c636e1dd67cedf8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 33207 } }
I20260620 09:25:27.655990 23462 leader_election.cc:288] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [CANDIDATE]: Term 1 election: Requested vote from peers f8cf09b32dc94f6081112fc10bff10db (127.4.59.3:45055), ebc4fe782cec46679c636e1dd67cedf8 (127.4.59.2:33207)
I20260620 09:25:27.656118 23394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77b893d9e4a04007b9f3465437530284" candidate_uuid: "3681775a4f83438ab741263e5996b97c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8cf09b32dc94f6081112fc10bff10db"
I20260620 09:25:27.656159 23263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77b893d9e4a04007b9f3465437530284" candidate_uuid: "3681775a4f83438ab741263e5996b97c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebc4fe782cec46679c636e1dd67cedf8"
I20260620 09:25:27.656217 23394 raft_consensus.cc:3037] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:27.656217 23263 raft_consensus.cc:3037] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:27.656955 23394 raft_consensus.cc:2454] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3681775a4f83438ab741263e5996b97c in term 1.
I20260620 09:25:27.657130 23263 raft_consensus.cc:2454] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3681775a4f83438ab741263e5996b97c in term 1.
I20260620 09:25:27.657122 23065 leader_election.cc:302] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3681775a4f83438ab741263e5996b97c, f8cf09b32dc94f6081112fc10bff10db; no voters:
I20260620 09:25:27.657235 23462 raft_consensus.cc:2781] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:27.657369 23462 raft_consensus.cc:686] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [term 1 LEADER]: Becoming Leader. State: Replica: 3681775a4f83438ab741263e5996b97c, State: Running, Role: LEADER
I20260620 09:25:27.657514 23462 consensus_queue.cc:237] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3681775a4f83438ab741263e5996b97c" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46159 } } peers { permanent_uuid: "f8cf09b32dc94f6081112fc10bff10db" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 45055 } } peers { permanent_uuid: "ebc4fe782cec46679c636e1dd67cedf8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 33207 } }
I20260620 09:25:27.658414 22990 catalog_manager.cc:5495] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3681775a4f83438ab741263e5996b97c (127.4.59.1). New cstate: current_term: 1 leader_uuid: "3681775a4f83438ab741263e5996b97c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3681775a4f83438ab741263e5996b97c" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 46159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8cf09b32dc94f6081112fc10bff10db" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 45055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebc4fe782cec46679c636e1dd67cedf8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 33207 } health_report { overall_health: UNKNOWN } } }
W20260620 09:25:27.670935 23310 tablet.cc:2369] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:25:27.685956 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.683699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58582 (local address 127.4.59.1:46159)
0620 09:25:27.683744 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:27.683746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:27.683753 (+ 7us) server_negotiation.cc:407] Connection header received
0620 09:25:27.683812 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:27.683812 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:27.683830 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:27.683894 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:25:27.684019 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.684465 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.685298 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:27.685488 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:27.685546 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:27.685673 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:25:27.685754 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:25:27.685821 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:27.685866 (+ 45us) server_negotiation.cc:299] Negotiation successful
0620 09:25:27.685875 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 09:25:27.687705 23263 raft_consensus.cc:1261] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8 [term 1 FOLLOWER]: Refusing update from remote peer 3681775a4f83438ab741263e5996b97c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:25:27.687716 23394 raft_consensus.cc:1261] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db [term 1 FOLLOWER]: Refusing update from remote peer 3681775a4f83438ab741263e5996b97c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:25:27.688076 23464 consensus_queue.cc:1035] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8cf09b32dc94f6081112fc10bff10db" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 45055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:25:27.688191 23462 consensus_queue.cc:1035] T 77b893d9e4a04007b9f3465437530284 P 3681775a4f83438ab741263e5996b97c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebc4fe782cec46679c636e1dd67cedf8" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 33207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:25:27.690425 23467 mvcc.cc:205] Tried to move back new op lower bound from 7298857073406177280 to 7298857073285890048. Current Snapshot: MvccSnapshot[applied={T|T < 7298857073406177280}]
I20260620 09:25:27.690941 23469 mvcc.cc:205] Tried to move back new op lower bound from 7298857073406177280 to 7298857073285890048. Current Snapshot: MvccSnapshot[applied={T|T < 7298857073406177280 or (T in {7298857073406177280})}]
I20260620 09:25:27.691075 23468 mvcc.cc:205] Tried to move back new op lower bound from 7298857073406177280 to 7298857073285890048. Current Snapshot: MvccSnapshot[applied={T|T < 7298857073406177280}]
I20260620 09:25:27.692863 22990 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:25:27.692960 22990 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:25:27.693650 22990 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76227e9dd2a843d39753fb754ca49c76: Sending DeleteTablet for 3 replicas of tablet 77b893d9e4a04007b9f3465437530284
I20260620 09:25:27.693989 23374 tablet_service.cc:1514] Processing DeleteTablet for tablet 77b893d9e4a04007b9f3465437530284 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:27 UTC) from {username='slave'} at 127.0.0.1:41672
I20260620 09:25:27.694064 23112 tablet_service.cc:1514] Processing DeleteTablet for tablet 77b893d9e4a04007b9f3465437530284 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:27 UTC) from {username='slave'} at 127.0.0.1:58580
I20260620 09:25:27.694110 23243 tablet_service.cc:1514] Processing DeleteTablet for tablet 77b893d9e4a04007b9f3465437530284 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:27 UTC) from {username='slave'} at 127.0.0.1:37620
I20260620 09:25:27.694314 23479 tablet_replica.cc:330] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8: stopping tablet replica
I20260620 09:25:27.694408 23479 raft_consensus.cc:2229] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:25:27.694518 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 23050
I20260620 09:25:27.695376 23477 tablet_replica.cc:330] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db: stopping tablet replica
I20260620 09:25:27.695483 23477 raft_consensus.cc:2229] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:25:27.695597 23477 raft_consensus.cc:2258] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:25:27.695753 23479 raft_consensus.cc:2258] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:25:27.696026 23477 ts_tablet_manager.cc:1905] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:25:27.696225 23479 ts_tablet_manager.cc:1905] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:25:27.697685 23477 ts_tablet_manager.cc:1918] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:25:27.697702 23479 ts_tablet_manager.cc:1918] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:25:27.697755 23477 log.cc:1196] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/77b893d9e4a04007b9f3465437530284
I20260620 09:25:27.697783 23479 log.cc:1196] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/77b893d9e4a04007b9f3465437530284
I20260620 09:25:27.698024 23477 ts_tablet_manager.cc:1939] T 77b893d9e4a04007b9f3465437530284 P f8cf09b32dc94f6081112fc10bff10db: Deleting consensus metadata
I20260620 09:25:27.698040 23479 ts_tablet_manager.cc:1939] T 77b893d9e4a04007b9f3465437530284 P ebc4fe782cec46679c636e1dd67cedf8: Deleting consensus metadata
I20260620 09:25:27.698405 22974 catalog_manager.cc:4841] TS f8cf09b32dc94f6081112fc10bff10db (127.4.59.3:45055): tablet 77b893d9e4a04007b9f3465437530284 (table test-table [id=5f3422dae40f4673bdadffecd573ff29]) successfully deleted
I20260620 09:25:27.698436 22977 catalog_manager.cc:4841] TS ebc4fe782cec46679c636e1dd67cedf8 (127.4.59.2:33207): tablet 77b893d9e4a04007b9f3465437530284 (table test-table [id=5f3422dae40f4673bdadffecd573ff29]) successfully deleted
W20260620 09:25:27.701114 22977 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.59.1:46159 (error 108)
I20260620 09:25:27.701381 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 23181
I20260620 09:25:27.701617 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:27.701433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:46159 (local address 127.0.0.1:58584)
0620 09:25:27.701514 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:27.701580 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:46159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260620 09:25:27.701944 22977 catalog_manager.cc:4584] TS 3681775a4f83438ab741263e5996b97c (127.4.59.1:46159): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77b893d9e4a04007b9f3465437530284: Network error: Client connection negotiation failed: client connection to 127.4.59.1:46159: connect: Connection refused (error 111)
I20260620 09:25:27.706908 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 23312
I20260620 09:25:27.711244 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 22959
2026-06-20T09:25:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (710 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:27 dist-test-slave-414w krb5kdc[23483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:27 dist-test-slave-414w krb5kdc[23483](info): set up 2 sockets
Jun 20 09:25:27 dist-test-slave-414w krb5kdc[23483](info): commencing operation
krb5kdc: starting...
W20260620 09:25:28.741353 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:28 dist-test-slave-414w krb5kdc[23483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:25:28.852451 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:43657
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:43657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:28.931226 23496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:28.931409 23496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:28.931430 23496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:28.932561 23496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:28.932602 23496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:28.932616 23496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:28.932629 23496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:28.932641 23496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:28.934182 23496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:43657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:43657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:28.934425 23496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:28.934643 23496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:28.939498 23501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:28.939633 23496 server_base.cc:1034] running on GCE node
W20260620 09:25:28.939505 23502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:28.939545 23504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:28.939967 23496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:28.940266 23496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:28.941407 23496 hybrid_clock.cc:648] HybridClock initialized: now 1781947528941387 us; error 38 us; skew 500 ppm
Jun 20 09:25:28 dist-test-slave-414w krb5kdc[23483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947528, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:28.943974 23496 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:25:28.944427 23496 webserver.cc:460] Webserver started at http://127.4.59.62:46653/ using document root <none> and password file <none>
I20260620 09:25:28.944664 23496 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:28.944730 23496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:28.944880 23496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:28.945958 23496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "1779251b1eba4782b707a5aeb96353cd"
format_stamp: "Formatted at 2026-06-20 09:25:28 on dist-test-slave-414w"
server_key: "d4d663f1b3bff64888e4da4a31f71c0b"
server_key_iv: "96a5e8778334038196c1a35a37385f9f"
server_key_version: "encryptionkey@0"
I20260620 09:25:28.946305 23496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "1779251b1eba4782b707a5aeb96353cd"
format_stamp: "Formatted at 2026-06-20 09:25:28 on dist-test-slave-414w"
server_key: "d4d663f1b3bff64888e4da4a31f71c0b"
server_key_iv: "96a5e8778334038196c1a35a37385f9f"
server_key_version: "encryptionkey@0"
I20260620 09:25:28.947595 23496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 09:25:28.948313 23511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:28.948491 23496 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:25:28.948581 23496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "1779251b1eba4782b707a5aeb96353cd"
format_stamp: "Formatted at 2026-06-20 09:25:28 on dist-test-slave-414w"
server_key: "d4d663f1b3bff64888e4da4a31f71c0b"
server_key_iv: "96a5e8778334038196c1a35a37385f9f"
server_key_version: "encryptionkey@0"
I20260620 09:25:28.948662 23496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:28.968559 23496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:28.970696 23496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:28.970854 23496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:28.974589 23496 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:43657
I20260620 09:25:28.974642 23563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:43657 every 8 connection(s)
I20260620 09:25:28.974990 23496 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:28.975838 23564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:28.976569 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 23496
I20260620 09:25:28.976703 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:28.976893 4332 external_mini_cluster.cc:1428] Setting key fefc49db9995dc62a2cef0601bdd3621
I20260620 09:25:28.978955 23564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd: Bootstrap starting.
I20260620 09:25:28.979621 23564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:28.979928 23564 log.cc:824] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:28.980595 23564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd: No bootstrap required, opened a new log
I20260620 09:25:28.982707 23564 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1779251b1eba4782b707a5aeb96353cd" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 43657 } }
I20260620 09:25:28.982834 23564 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:28.982864 23564 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1779251b1eba4782b707a5aeb96353cd, State: Initialized, Role: FOLLOWER
I20260620 09:25:28.983012 23564 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1779251b1eba4782b707a5aeb96353cd" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 43657 } }
I20260620 09:25:28.983075 23564 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:28.983095 23564 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:28.983161 23564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:28.983671 23564 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1779251b1eba4782b707a5aeb96353cd" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 43657 } }
Jun 20 09:25:28 dist-test-slave-414w krb5kdc[23483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:28.983788 23564 leader_election.cc:302] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1779251b1eba4782b707a5aeb96353cd; no voters:
I20260620 09:25:28.984086 23564 leader_election.cc:288] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:28.984113 23569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:28.984295 23569 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [term 1 LEADER]: Becoming Leader. State: Replica: 1779251b1eba4782b707a5aeb96353cd, State: Running, Role: LEADER
I20260620 09:25:28.984366 23564 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:28.984552 23569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1779251b1eba4782b707a5aeb96353cd" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 43657 } }
I20260620 09:25:28.985643 23570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1779251b1eba4782b707a5aeb96353cd. Latest consensus state: current_term: 1 leader_uuid: "1779251b1eba4782b707a5aeb96353cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1779251b1eba4782b707a5aeb96353cd" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 43657 } } }
I20260620 09:25:28.985801 23570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:28.985926 23571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1779251b1eba4782b707a5aeb96353cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1779251b1eba4782b707a5aeb96353cd" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 43657 } } }
W20260620 09:25:28.986442 23584 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260620 09:25:28.986724 23571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd [sys.catalog]: This master's current role is: LEADER
W20260620 09:25:28.986734 23584 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:25:28.986836 23585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:28.986996 23585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:28.988292 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:28.977444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40938 (local address 127.4.59.62:43657)
0620 09:25:28.977626 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:28.977636 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:28.977672 (+ 36us) server_negotiation.cc:407] Connection header received
0620 09:25:28.978199 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:28.978207 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:28.978268 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:28.978423 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:28.978991 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:28.980281 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:28.981080 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:28.981730 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:28.981845 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:28.984259 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:28.984263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:28.984264 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:28.984277 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:28.986421 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:28.986983 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:28.986986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:28.986987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:28.987107 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:28.987325 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:28.987325 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:28.987326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:28.987612 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:28.987757 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:28.988027 (+ 270us) server_negotiation.cc:299] Negotiation successful
0620 09:25:28.988131 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":78,"threads_started":1}
I20260620 09:25:28.988641 23585 catalog_manager.cc:1347] Generated new cluster ID: f4431bb0c09540e98680b589cb098893
I20260620 09:25:28.988699 23585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:29.006526 23585 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:29.007292 23585 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:29.012890 23585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd: Generated new TSK 0
I20260620 09:25:29.013077 23585 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.59.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
Entry for principal kudu/127.4.59.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab.
I20260620 09:25:29.042173 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:43657
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:29.119365 23590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:29.119547 23590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:29.119570 23590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:29.120672 23590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:29.120719 23590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:29.120746 23590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:25:29.122154 23590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.1.keytab
--principal=kudu/127.4.59.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:43657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:29.122383 23590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:29.122615 23590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:29.127430 23598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:29.127465 23596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:29.127595 23590 server_base.cc:1034] running on GCE node
W20260620 09:25:29.127638 23595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:29.127831 23590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:29.128085 23590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:29.129202 23590 hybrid_clock.cc:648] HybridClock initialized: now 1781947529129203 us; error 43 us; skew 500 ppm
Jun 20 09:25:29 dist-test-slave-414w krb5kdc[23483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947529, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:29.131537 23590 init.cc:377] Logged in from keytab as kudu/127.4.59.1@KRBTEST.COM (short username kudu)
I20260620 09:25:29.131937 23590 webserver.cc:460] Webserver started at http://127.4.59.1:38041/ using document root <none> and password file <none>
I20260620 09:25:29.132161 23590 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:29.132225 23590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:29.132369 23590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:29.133419 23590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "142b019a37fe46989a156ebbb483eaf7"
format_stamp: "Formatted at 2026-06-20 09:25:29 on dist-test-slave-414w"
server_key: "69a59797650e4faaad8d790eff6281dc"
server_key_iv: "9f533aa8869103f9c9183b6af51a8463"
server_key_version: "encryptionkey@0"
I20260620 09:25:29.133775 23590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "142b019a37fe46989a156ebbb483eaf7"
format_stamp: "Formatted at 2026-06-20 09:25:29 on dist-test-slave-414w"
server_key: "69a59797650e4faaad8d790eff6281dc"
server_key_iv: "9f533aa8869103f9c9183b6af51a8463"
server_key_version: "encryptionkey@0"
I20260620 09:25:29.135035 23590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:25:29.135746 23605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:29.135949 23590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:29.136019 23590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "142b019a37fe46989a156ebbb483eaf7"
format_stamp: "Formatted at 2026-06-20 09:25:29 on dist-test-slave-414w"
server_key: "69a59797650e4faaad8d790eff6281dc"
server_key_iv: "9f533aa8869103f9c9183b6af51a8463"
server_key_version: "encryptionkey@0"
I20260620 09:25:29.136106 23590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:29.158456 23590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:29.160730 23590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:29.160869 23590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:29.161109 23590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:29.161442 23590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:29.161476 23590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:29.161499 23590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:29.161511 23590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:29.166988 23590 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:38223
I20260620 09:25:29.167030 23718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:38223 every 8 connection(s)
I20260620 09:25:29.167315 23590 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
Jun 20 09:25:29 dist-test-slave-414w krb5kdc[23483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947529, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.1@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:29.175861 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.167996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:56125 (local address 127.4.59.62:43657)
0620 09:25:29.168040 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:29.168042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:29.168209 (+ 167us) server_negotiation.cc:407] Connection header received
0620 09:25:29.168685 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:29.168686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:29.168696 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:29.168767 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:29.169682 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.170176 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.171082 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.171272 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.171321 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.173283 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:29.173284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:29.173285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:29.173291 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:29.174766 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:29.175110 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:29.175111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:29.175111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:29.175159 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:29.175354 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:29.175355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:29.175355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:29.175449 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:29.175497 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:29.175727 (+ 230us) server_negotiation.cc:299] Negotiation successful
0620 09:25:29.175795 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 09:25:29.175989 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.167944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:43657 (local address 127.4.59.1:56125)
0620 09:25:29.168098 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:29.168139 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:29.168598 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:29.168815 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:29.168818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:29.169424 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:29.169606 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.169611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.170254 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.170254 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.171009 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.171010 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.171342 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.171343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.171446 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.171854 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:29.171863 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:29.173194 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:29.174839 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:29.174844 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:29.174846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:29.175046 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:29.175215 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:29.175215 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:29.175216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:29.175303 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:29.175501 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:29.175505 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:29.175609 (+ 104us) client_negotiation.cc:769] Sending connection context
0620 09:25:29.175685 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 09:25:29.175793 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260620 09:25:29.176364 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 23590
I20260620 09:25:29.176482 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:25:29.176506 23719 heartbeater.cc:347] Connected to a master server at 127.4.59.62:43657
I20260620 09:25:29.176599 23719 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:29.176680 4332 external_mini_cluster.cc:1428] Setting key 438fbdbd4f24658087a75324d548abf6
I20260620 09:25:29.176793 23719 heartbeater.cc:511] Master 127.4.59.62:43657 requested a full tablet report, sending...
I20260620 09:25:29.177348 23527 ts_manager.cc:194] Registered new tserver with Master: 142b019a37fe46989a156ebbb483eaf7 (127.4.59.1:38223)
I20260620 09:25:29.177932 23527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.1@KRBTEST.COM'} at 127.4.59.1:56125
WARNING: no policy specified for kudu/127.4.59.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
Entry for principal kudu/127.4.59.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab.
I20260620 09:25:29.200115 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:43657
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:29.278764 23724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:29.278952 23724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:29.278976 23724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:29.280014 23724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:29.280048 23724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:29.280074 23724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:25:29.281417 23724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.2.keytab
--principal=kudu/127.4.59.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:43657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:29.281585 23724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:29.281744 23724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:29.286620 23732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:29.286605 23729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:29.286667 23730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:29.286960 23724 server_base.cc:1034] running on GCE node
I20260620 09:25:29.287206 23724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:29.287454 23724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:29.288599 23724 hybrid_clock.cc:648] HybridClock initialized: now 1781947529288569 us; error 46 us; skew 500 ppm
Jun 20 09:25:29 dist-test-slave-414w krb5kdc[23483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947529, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:29.291011 23724 init.cc:377] Logged in from keytab as kudu/127.4.59.2@KRBTEST.COM (short username kudu)
I20260620 09:25:29.291461 23724 webserver.cc:460] Webserver started at http://127.4.59.2:36307/ using document root <none> and password file <none>
I20260620 09:25:29.291676 23724 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:29.291740 23724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:29.291898 23724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:29.292893 23724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "9d365669c36f4e3d8d8a4f82cc5c5200"
format_stamp: "Formatted at 2026-06-20 09:25:29 on dist-test-slave-414w"
server_key: "cff5047ccdd959b8888fab6c35a8350b"
server_key_iv: "c49ce9d280aff24bda2ce650fa8d1b6b"
server_key_version: "encryptionkey@0"
I20260620 09:25:29.293278 23724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "9d365669c36f4e3d8d8a4f82cc5c5200"
format_stamp: "Formatted at 2026-06-20 09:25:29 on dist-test-slave-414w"
server_key: "cff5047ccdd959b8888fab6c35a8350b"
server_key_iv: "c49ce9d280aff24bda2ce650fa8d1b6b"
server_key_version: "encryptionkey@0"
I20260620 09:25:29.294625 23724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:25:29.295312 23739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:29.295467 23724 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:25:29.295560 23724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "9d365669c36f4e3d8d8a4f82cc5c5200"
format_stamp: "Formatted at 2026-06-20 09:25:29 on dist-test-slave-414w"
server_key: "cff5047ccdd959b8888fab6c35a8350b"
server_key_iv: "c49ce9d280aff24bda2ce650fa8d1b6b"
server_key_version: "encryptionkey@0"
I20260620 09:25:29.295639 23724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:29.306732 23724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:29.308555 23724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:29.308706 23724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:29.308928 23724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:29.309279 23724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:29.309333 23724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:29.309368 23724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:29.309396 23724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:29.315096 23724 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:41145
I20260620 09:25:29.315150 23852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:41145 every 8 connection(s)
I20260620 09:25:29.315443 23724 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
Jun 20 09:25:29 dist-test-slave-414w krb5kdc[23483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947529, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.2@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:29.324187 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.316111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:37123 (local address 127.4.59.62:43657)
0620 09:25:29.316151 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:29.316154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:29.316343 (+ 189us) server_negotiation.cc:407] Connection header received
0620 09:25:29.316826 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:29.316827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:29.316839 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:29.316914 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:29.317862 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.318362 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.319319 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.319506 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.319562 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.321524 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:29.321525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:29.321526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:29.321532 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:29.322941 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:29.323285 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:29.323286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:29.323286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:29.323335 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:29.323622 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:29.323623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:29.323623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:29.323738 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:29.323843 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:29.324058 (+ 215us) server_negotiation.cc:299] Negotiation successful
0620 09:25:29.324124 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 09:25:29.324309 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 23724
I20260620 09:25:29.324433 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:25:29.324486 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.316073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:43657 (local address 127.4.59.2:37123)
0620 09:25:29.316219 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:29.316259 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:29.316718 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:29.316963 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:29.316966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:29.317588 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:29.317774 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.317779 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.318443 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.318443 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.319238 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.319240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.319579 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.319580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.319671 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.320075 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:29.320083 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:29.321416 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:29.323015 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:29.323020 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:29.323022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:29.323219 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:29.323416 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:29.323417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:29.323417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:29.323531 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:29.323820 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:29.323825 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:29.323939 (+ 114us) client_negotiation.cc:769] Sending connection context
0620 09:25:29.324020 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 09:25:29.324235 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":84,"threads_started":1}
I20260620 09:25:29.324631 4332 external_mini_cluster.cc:1428] Setting key e5df2e56e7f37392a2a581461f821f21
I20260620 09:25:29.324962 23853 heartbeater.cc:347] Connected to a master server at 127.4.59.62:43657
I20260620 09:25:29.325064 23853 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:29.325266 23853 heartbeater.cc:511] Master 127.4.59.62:43657 requested a full tablet report, sending...
I20260620 09:25:29.325639 23527 ts_manager.cc:194] Registered new tserver with Master: 9d365669c36f4e3d8d8a4f82cc5c5200 (127.4.59.2:41145)
I20260620 09:25:29.326020 23527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.2@KRBTEST.COM'} at 127.4.59.2:37123
WARNING: no policy specified for kudu/127.4.59.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
Entry for principal kudu/127.4.59.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab.
I20260620 09:25:29.351719 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:43657
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:29.432410 23858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:29.432610 23858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:29.432633 23858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:29.433748 23858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:29.433787 23858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:29.433813 23858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:25:29.435145 23858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.3.keytab
--principal=kudu/127.4.59.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:43657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:29.435320 23858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:29.435487 23858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:29.440500 23863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:29.440634 23858 server_base.cc:1034] running on GCE node
W20260620 09:25:29.440502 23864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:29.440481 23866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:29.440917 23858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:29.441218 23858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:29.442399 23858 hybrid_clock.cc:648] HybridClock initialized: now 1781947529442390 us; error 38 us; skew 500 ppm
Jun 20 09:25:29 dist-test-slave-414w krb5kdc[23483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947529, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:29.444890 23858 init.cc:377] Logged in from keytab as kudu/127.4.59.3@KRBTEST.COM (short username kudu)
I20260620 09:25:29.445380 23858 webserver.cc:460] Webserver started at http://127.4.59.3:35931/ using document root <none> and password file <none>
I20260620 09:25:29.445603 23858 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:29.445675 23858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:29.445811 23858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:29.446825 23858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "61cea6ef595c42ec9a89377482ef4a06"
format_stamp: "Formatted at 2026-06-20 09:25:29 on dist-test-slave-414w"
server_key: "44031b9e3ca9dc490953662768351844"
server_key_iv: "8feab85a215e8800e8b90113640191ed"
server_key_version: "encryptionkey@0"
I20260620 09:25:29.447187 23858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "61cea6ef595c42ec9a89377482ef4a06"
format_stamp: "Formatted at 2026-06-20 09:25:29 on dist-test-slave-414w"
server_key: "44031b9e3ca9dc490953662768351844"
server_key_iv: "8feab85a215e8800e8b90113640191ed"
server_key_version: "encryptionkey@0"
I20260620 09:25:29.449589 23858 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260620 09:25:29.450887 23873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:29.451094 23858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:29.451185 23858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "61cea6ef595c42ec9a89377482ef4a06"
format_stamp: "Formatted at 2026-06-20 09:25:29 on dist-test-slave-414w"
server_key: "44031b9e3ca9dc490953662768351844"
server_key_iv: "8feab85a215e8800e8b90113640191ed"
server_key_version: "encryptionkey@0"
I20260620 09:25:29.451239 23858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:29.461066 23858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:29.462972 23858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:29.463081 23858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:29.463246 23858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:29.463485 23858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:29.463513 23858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:29.463534 23858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:29.463547 23858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:29.469054 23858 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:41507
I20260620 09:25:29.469102 23986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:41507 every 8 connection(s)
I20260620 09:25:29.469419 23858 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
Jun 20 09:25:29 dist-test-slave-414w krb5kdc[23483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947529, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.3@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:29.476691 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 23858
I20260620 09:25:29.476815 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:25:29.476989 4332 external_mini_cluster.cc:1428] Setting key 6e2931b41683f66323794c0d421f326e
I20260620 09:25:29.478837 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.470104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:57549 (local address 127.4.59.62:43657)
0620 09:25:29.470205 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:29.470207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:29.470406 (+ 199us) server_negotiation.cc:407] Connection header received
0620 09:25:29.470916 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:29.470917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:29.470927 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:29.470979 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:29.471842 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.472459 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.473438 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.473622 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.473684 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.475775 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:29.475776 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:29.475776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:29.475781 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:29.477379 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:29.477900 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:29.477901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:29.477901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:29.477959 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:29.478217 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:29.478218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:29.478219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:29.478333 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:29.478412 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:29.478671 (+ 259us) server_negotiation.cc:299] Negotiation successful
0620 09:25:29.478744 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260620 09:25:29.479028 23989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.470081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:43657 (local address 127.4.59.3:57549)
0620 09:25:29.470254 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:29.470292 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:29.470817 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:29.471027 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:29.471030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:29.471570 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:29.471762 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.471766 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.472564 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.472564 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.473364 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.473367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.473747 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.473748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.473861 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.474330 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:29.474338 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:29.475698 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:29.477457 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:29.477464 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:29.477466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:29.477820 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:29.478034 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:29.478035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:29.478036 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:29.478146 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:29.478403 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:29.478409 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:29.478544 (+ 135us) client_negotiation.cc:769] Sending connection context
0620 09:25:29.478650 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 09:25:29.478808 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260620 09:25:29.479460 23987 heartbeater.cc:347] Connected to a master server at 127.4.59.62:43657
I20260620 09:25:29.479552 23987 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:29.479718 23987 heartbeater.cc:511] Master 127.4.59.62:43657 requested a full tablet report, sending...
I20260620 09:25:29.480155 23527 ts_manager.cc:194] Registered new tserver with Master: 61cea6ef595c42ec9a89377482ef4a06 (127.4.59.3:41507)
I20260620 09:25:29.480634 23527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.59.3@KRBTEST.COM'} at 127.4.59.3:57549
I20260620 09:25:29.488817 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:25:29.495874 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.490221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54946 (local address 127.4.59.62:43657)
0620 09:25:29.490286 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:29.490288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:29.490360 (+ 72us) server_negotiation.cc:407] Connection header received
0620 09:25:29.490401 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:29.490402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:29.490410 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:29.490456 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:29.490982 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.491436 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.492140 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.492313 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.492363 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.493572 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:29.493573 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:29.493573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:29.493579 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:29.494860 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:29.495176 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:29.495177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:29.495177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:29.495225 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:29.495397 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:29.495397 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:29.495397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:29.495531 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:29.495604 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:29.495703 (+ 99us) server_negotiation.cc:299] Negotiation successful
0620 09:25:29.495759 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 09:25:29.497299 23527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:25:29.498972 23527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:25:29.505009 24000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.501210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56940 (local address 127.4.59.1:38223)
0620 09:25:29.501411 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:29.501415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:29.501482 (+ 67us) server_negotiation.cc:407] Connection header received
0620 09:25:29.501559 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:29.501562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:29.501579 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:29.501661 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:29.501769 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.502870 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.504090 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.504577 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.504663 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.504723 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:29.504877 (+ 154us) server_negotiation.cc:299] Negotiation successful
0620 09:25:29.504920 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":97,"threads_started":1}
I20260620 09:25:29.505007 23997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.500965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:38223 (local address 127.0.0.1:56940)
0620 09:25:29.501390 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:29.501400 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:29.501488 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:29.501657 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:29.501657 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:29.501660 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:29.501703 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.501705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.502978 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.502979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.504006 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.504008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.504662 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.504663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.504747 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.504760 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 09:25:29.504846 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 09:25:29.504902 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":1,"thread_start_us":54,"threads_started":1}
I20260620 09:25:29.505028 23996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.500949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:41507 (local address 127.0.0.1:57570)
0620 09:25:29.501103 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:29.501165 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:29.501224 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:29.501568 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:29.501572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:29.501575 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:29.501638 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.501640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.502610 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.502611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.503814 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.503817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.504757 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.504758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.504827 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.504832 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 09:25:29.504863 (+ 31us) client_negotiation.cc:240] Negotiation successful
0620 09:25:29.504926 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":25,"thread_start_us":51,"threads_started":1}
I20260620 09:25:29.505196 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.500965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:41145 (local address 127.0.0.1:40192)
0620 09:25:29.501568 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:29.501577 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:29.501620 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:29.501833 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:29.501834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:29.501836 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:29.501873 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.501874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.503188 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.503189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.504005 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.504008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.504943 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.504944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.505000 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.505004 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:25:29.505051 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 09:25:29.505088 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":10,"thread_start_us":45,"threads_started":1}
I20260620 09:25:29.505461 23999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.501206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40192 (local address 127.4.59.2:41145)
0620 09:25:29.501499 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:29.501502 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:29.501681 (+ 179us) server_negotiation.cc:407] Connection header received
0620 09:25:29.501731 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:29.501733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:29.501748 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:29.501823 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:29.501929 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.502964 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.504107 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.504785 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.504856 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.504911 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:29.505337 (+ 426us) server_negotiation.cc:299] Negotiation successful
0620 09:25:29.505381 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":104,"threads_started":1}
I20260620 09:25:29.505522 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.501028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57570 (local address 127.4.59.3:41507)
0620 09:25:29.501379 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:29.501385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:29.501395 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:25:29.501435 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:29.501437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:29.501453 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:29.501558 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:29.501723 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.502500 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.503916 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.504655 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.505291 (+ 636us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.505369 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:29.505394 (+ 25us) server_negotiation.cc:299] Negotiation successful
0620 09:25:29.505435 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":77,"threads_started":1}
I20260620 09:25:29.506172 23787 tablet_service.cc:1467] Processing CreateTablet for tablet 56025e93f7184833a10c8e58a67b9055 (DEFAULT_TABLE table=test-table [id=bc85398b36a24365a40f0737ac9f5e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:29.506371 23653 tablet_service.cc:1467] Processing CreateTablet for tablet 56025e93f7184833a10c8e58a67b9055 (DEFAULT_TABLE table=test-table [id=bc85398b36a24365a40f0737ac9f5e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:29.506381 23921 tablet_service.cc:1467] Processing CreateTablet for tablet 56025e93f7184833a10c8e58a67b9055 (DEFAULT_TABLE table=test-table [id=bc85398b36a24365a40f0737ac9f5e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:29.506547 23787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56025e93f7184833a10c8e58a67b9055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:29.506685 23921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56025e93f7184833a10c8e58a67b9055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:29.506713 23653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56025e93f7184833a10c8e58a67b9055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:29.509335 24001 tablet_bootstrap.cc:492] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7: Bootstrap starting.
I20260620 09:25:29.509774 24003 tablet_bootstrap.cc:492] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06: Bootstrap starting.
I20260620 09:25:29.510423 24001 tablet_bootstrap.cc:654] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:29.510423 24003 tablet_bootstrap.cc:654] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:29.511031 24003 log.cc:824] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:29.511034 24001 log.cc:824] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:29.511372 24002 tablet_bootstrap.cc:492] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200: Bootstrap starting.
I20260620 09:25:29.511829 24003 tablet_bootstrap.cc:492] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06: No bootstrap required, opened a new log
I20260620 09:25:29.511916 24003 ts_tablet_manager.cc:1397] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:25:29.511987 24002 tablet_bootstrap.cc:654] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:29.512523 24002 log.cc:824] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:29.513278 24002 tablet_bootstrap.cc:492] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200: No bootstrap required, opened a new log
I20260620 09:25:29.513478 24002 ts_tablet_manager.cc:1397] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:25:29.513486 24001 tablet_bootstrap.cc:492] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7: No bootstrap required, opened a new log
I20260620 09:25:29.513550 24001 ts_tablet_manager.cc:1397] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 09:25:29.514050 24003 raft_consensus.cc:348] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d365669c36f4e3d8d8a4f82cc5c5200" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 41145 } } peers { permanent_uuid: "61cea6ef595c42ec9a89377482ef4a06" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41507 } } peers { permanent_uuid: "142b019a37fe46989a156ebbb483eaf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38223 } }
I20260620 09:25:29.514233 24003 raft_consensus.cc:374] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:29.514268 24003 raft_consensus.cc:729] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61cea6ef595c42ec9a89377482ef4a06, State: Initialized, Role: FOLLOWER
I20260620 09:25:29.514406 24003 consensus_queue.cc:260] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d365669c36f4e3d8d8a4f82cc5c5200" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 41145 } } peers { permanent_uuid: "61cea6ef595c42ec9a89377482ef4a06" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41507 } } peers { permanent_uuid: "142b019a37fe46989a156ebbb483eaf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38223 } }
I20260620 09:25:29.514674 24003 ts_tablet_manager.cc:1428] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 09:25:29.514817 23987 heartbeater.cc:503] Master 127.4.59.62:43657 was elected leader, sending a full tablet report...
I20260620 09:25:29.515389 24002 raft_consensus.cc:348] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d365669c36f4e3d8d8a4f82cc5c5200" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 41145 } } peers { permanent_uuid: "61cea6ef595c42ec9a89377482ef4a06" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41507 } } peers { permanent_uuid: "142b019a37fe46989a156ebbb483eaf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38223 } }
I20260620 09:25:29.515506 24002 raft_consensus.cc:374] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:29.515554 24002 raft_consensus.cc:729] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d365669c36f4e3d8d8a4f82cc5c5200, State: Initialized, Role: FOLLOWER
I20260620 09:25:29.515476 24001 raft_consensus.cc:348] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d365669c36f4e3d8d8a4f82cc5c5200" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 41145 } } peers { permanent_uuid: "61cea6ef595c42ec9a89377482ef4a06" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41507 } } peers { permanent_uuid: "142b019a37fe46989a156ebbb483eaf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38223 } }
I20260620 09:25:29.515648 24001 raft_consensus.cc:374] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:29.515656 24002 consensus_queue.cc:260] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d365669c36f4e3d8d8a4f82cc5c5200" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 41145 } } peers { permanent_uuid: "61cea6ef595c42ec9a89377482ef4a06" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41507 } } peers { permanent_uuid: "142b019a37fe46989a156ebbb483eaf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38223 } }
I20260620 09:25:29.515712 24001 raft_consensus.cc:729] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142b019a37fe46989a156ebbb483eaf7, State: Initialized, Role: FOLLOWER
I20260620 09:25:29.515829 24001 consensus_queue.cc:260] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d365669c36f4e3d8d8a4f82cc5c5200" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 41145 } } peers { permanent_uuid: "61cea6ef595c42ec9a89377482ef4a06" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41507 } } peers { permanent_uuid: "142b019a37fe46989a156ebbb483eaf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38223 } }
I20260620 09:25:29.515936 24002 ts_tablet_manager.cc:1428] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:25:29.515964 23853 heartbeater.cc:503] Master 127.4.59.62:43657 was elected leader, sending a full tablet report...
I20260620 09:25:29.516096 24001 ts_tablet_manager.cc:1428] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 09:25:29.516245 23719 heartbeater.cc:503] Master 127.4.59.62:43657 was elected leader, sending a full tablet report...
W20260620 09:25:29.565985 23854 tablet.cc:2369] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:25:29.591688 24009 raft_consensus.cc:482] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:25:29.591809 24009 raft_consensus.cc:504] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d365669c36f4e3d8d8a4f82cc5c5200" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 41145 } } peers { permanent_uuid: "61cea6ef595c42ec9a89377482ef4a06" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41507 } } peers { permanent_uuid: "142b019a37fe46989a156ebbb483eaf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38223 } }
I20260620 09:25:29.592377 24009 leader_election.cc:288] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d365669c36f4e3d8d8a4f82cc5c5200 (127.4.59.2:41145), 61cea6ef595c42ec9a89377482ef4a06 (127.4.59.3:41507)
I20260620 09:25:29.596125 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.592657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:40119 (local address 127.4.59.3:41507)
0620 09:25:29.592733 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:29.592735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:29.592910 (+ 175us) server_negotiation.cc:407] Connection header received
0620 09:25:29.592992 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:29.592993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:29.593004 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:29.593077 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:29.593278 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.593937 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.595176 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.595738 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.595811 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.595821 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:29.596015 (+ 194us) server_negotiation.cc:299] Negotiation successful
0620 09:25:29.596055 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 09:25:29.596125 23999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.592653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:56897 (local address 127.4.59.2:41145)
0620 09:25:29.592696 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:29.592698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:29.592707 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:25:29.592749 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:29.592750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:29.592761 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:29.592845 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:29.593000 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.593766 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.595177 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.595738 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.595811 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.595821 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:29.596015 (+ 194us) server_negotiation.cc:299] Negotiation successful
0620 09:25:29.596055 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 09:25:29.596161 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.592578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:41145 (local address 127.4.59.1:56897)
0620 09:25:29.592625 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:29.592638 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:29.592680 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:29.592844 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:29.592845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:29.592846 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:29.592927 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.592929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.593853 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.593854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.595038 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.595040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.595829 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.595830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.595930 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.595932 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:25:29.595995 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 09:25:29.596059 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":5}
I20260620 09:25:29.596167 24010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.592578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:41507 (local address 127.4.59.1:40119)
0620 09:25:29.592838 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:29.592847 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:29.592925 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:29.593078 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:29.593079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:29.593081 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:29.593174 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.593176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.594005 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.594005 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.595078 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.595080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.595829 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.595830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.595927 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.595932 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 09:25:29.595995 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 09:25:29.596059 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":104,"thread_start_us":112,"threads_started":1}
I20260620 09:25:29.596546 23807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56025e93f7184833a10c8e58a67b9055" candidate_uuid: "142b019a37fe46989a156ebbb483eaf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d365669c36f4e3d8d8a4f82cc5c5200" is_pre_election: true
I20260620 09:25:29.596717 23807 raft_consensus.cc:2454] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 142b019a37fe46989a156ebbb483eaf7 in term 0.
I20260620 09:25:29.596720 23941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56025e93f7184833a10c8e58a67b9055" candidate_uuid: "142b019a37fe46989a156ebbb483eaf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61cea6ef595c42ec9a89377482ef4a06" is_pre_election: true
I20260620 09:25:29.596882 23941 raft_consensus.cc:2454] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 142b019a37fe46989a156ebbb483eaf7 in term 0.
I20260620 09:25:29.596940 23606 leader_election.cc:302] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142b019a37fe46989a156ebbb483eaf7, 9d365669c36f4e3d8d8a4f82cc5c5200; no voters:
I20260620 09:25:29.597074 24009 raft_consensus.cc:2781] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:25:29.597127 24009 raft_consensus.cc:482] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:25:29.597193 24009 raft_consensus.cc:3037] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:29.597898 24009 raft_consensus.cc:504] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d365669c36f4e3d8d8a4f82cc5c5200" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 41145 } } peers { permanent_uuid: "61cea6ef595c42ec9a89377482ef4a06" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41507 } } peers { permanent_uuid: "142b019a37fe46989a156ebbb483eaf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38223 } }
I20260620 09:25:29.598068 24009 leader_election.cc:288] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [CANDIDATE]: Term 1 election: Requested vote from peers 9d365669c36f4e3d8d8a4f82cc5c5200 (127.4.59.2:41145), 61cea6ef595c42ec9a89377482ef4a06 (127.4.59.3:41507)
I20260620 09:25:29.598217 23807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56025e93f7184833a10c8e58a67b9055" candidate_uuid: "142b019a37fe46989a156ebbb483eaf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d365669c36f4e3d8d8a4f82cc5c5200"
I20260620 09:25:29.598290 23807 raft_consensus.cc:3037] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:29.598435 23941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56025e93f7184833a10c8e58a67b9055" candidate_uuid: "142b019a37fe46989a156ebbb483eaf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61cea6ef595c42ec9a89377482ef4a06"
I20260620 09:25:29.598527 23941 raft_consensus.cc:3037] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:29.599073 23807 raft_consensus.cc:2454] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142b019a37fe46989a156ebbb483eaf7 in term 1.
I20260620 09:25:29.599233 23941 raft_consensus.cc:2454] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142b019a37fe46989a156ebbb483eaf7 in term 1.
I20260620 09:25:29.599432 23608 leader_election.cc:302] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142b019a37fe46989a156ebbb483eaf7, 61cea6ef595c42ec9a89377482ef4a06; no voters:
I20260620 09:25:29.599536 24009 raft_consensus.cc:2781] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:29.599711 24009 raft_consensus.cc:686] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [term 1 LEADER]: Becoming Leader. State: Replica: 142b019a37fe46989a156ebbb483eaf7, State: Running, Role: LEADER
I20260620 09:25:29.599810 24009 consensus_queue.cc:237] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d365669c36f4e3d8d8a4f82cc5c5200" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 41145 } } peers { permanent_uuid: "61cea6ef595c42ec9a89377482ef4a06" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41507 } } peers { permanent_uuid: "142b019a37fe46989a156ebbb483eaf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38223 } }
I20260620 09:25:29.600678 23528 catalog_manager.cc:5495] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 142b019a37fe46989a156ebbb483eaf7 (127.4.59.1). New cstate: current_term: 1 leader_uuid: "142b019a37fe46989a156ebbb483eaf7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d365669c36f4e3d8d8a4f82cc5c5200" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 41145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61cea6ef595c42ec9a89377482ef4a06" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142b019a37fe46989a156ebbb483eaf7" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 38223 } health_report { overall_health: HEALTHY } } }
I20260620 09:25:29.620736 24000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.618433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56942 (local address 127.4.59.1:38223)
0620 09:25:29.618480 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:29.618482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:29.618493 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:25:29.618528 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:29.618529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:29.618547 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:29.618613 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:25:29.618752 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.619209 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.619970 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.620173 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.620256 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.620428 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:25:29.620525 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:25:29.620616 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:29.620635 (+ 19us) server_negotiation.cc:299] Negotiation successful
0620 09:25:29.620655 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:25:29.622418 23807 raft_consensus.cc:1261] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200 [term 1 FOLLOWER]: Refusing update from remote peer 142b019a37fe46989a156ebbb483eaf7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:25:29.622429 23941 raft_consensus.cc:1261] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06 [term 1 FOLLOWER]: Refusing update from remote peer 142b019a37fe46989a156ebbb483eaf7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:25:29.622731 24009 consensus_queue.cc:1035] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61cea6ef595c42ec9a89377482ef4a06" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 41507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:25:29.622871 24011 consensus_queue.cc:1035] T 56025e93f7184833a10c8e58a67b9055 P 142b019a37fe46989a156ebbb483eaf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d365669c36f4e3d8d8a4f82cc5c5200" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 41145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:25:29.625816 24015 mvcc.cc:205] Tried to move back new op lower bound from 7298857081331089408 to 7298857081241698304. Current Snapshot: MvccSnapshot[applied={T|T < 7298857081331089408}]
I20260620 09:25:29.627369 23528 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:25:29.627457 23528 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:25:29.628223 23528 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1779251b1eba4782b707a5aeb96353cd: Sending DeleteTablet for 3 replicas of tablet 56025e93f7184833a10c8e58a67b9055
I20260620 09:25:29.628556 23653 tablet_service.cc:1514] Processing DeleteTablet for tablet 56025e93f7184833a10c8e58a67b9055 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:29 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:56940
I20260620 09:25:29.628594 23787 tablet_service.cc:1514] Processing DeleteTablet for tablet 56025e93f7184833a10c8e58a67b9055 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:29 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:40192
I20260620 09:25:29.628798 23921 tablet_service.cc:1514] Processing DeleteTablet for tablet 56025e93f7184833a10c8e58a67b9055 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:29 UTC) from {username='kudu', principal='kudu/127.4.59.62@KRBTEST.COM'} at 127.0.0.1:57570
I20260620 09:25:29.628983 24026 tablet_replica.cc:330] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06: stopping tablet replica
I20260620 09:25:29.629076 24026 raft_consensus.cc:2229] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:25:29.629094 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 23590
I20260620 09:25:29.630997 24026 raft_consensus.cc:2258] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:25:29.631160 24024 tablet_replica.cc:330] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200: stopping tablet replica
I20260620 09:25:29.631268 24024 raft_consensus.cc:2229] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:25:29.631366 24024 raft_consensus.cc:2258] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:25:29.631577 24026 ts_tablet_manager.cc:1905] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:25:29.631726 24024 ts_tablet_manager.cc:1905] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:25:29.633111 24026 ts_tablet_manager.cc:1918] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:25:29.633237 24026 log.cc:1196] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/56025e93f7184833a10c8e58a67b9055
I20260620 09:25:29.633510 24026 ts_tablet_manager.cc:1939] T 56025e93f7184833a10c8e58a67b9055 P 61cea6ef595c42ec9a89377482ef4a06: Deleting consensus metadata
I20260620 09:25:29.633594 24024 ts_tablet_manager.cc:1918] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:25:29.633675 24024 log.cc:1196] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/56025e93f7184833a10c8e58a67b9055
I20260620 09:25:29.633944 24024 ts_tablet_manager.cc:1939] T 56025e93f7184833a10c8e58a67b9055 P 9d365669c36f4e3d8d8a4f82cc5c5200: Deleting consensus metadata
I20260620 09:25:29.633922 23514 catalog_manager.cc:4841] TS 61cea6ef595c42ec9a89377482ef4a06 (127.4.59.3:41507): tablet 56025e93f7184833a10c8e58a67b9055 (table test-table [id=bc85398b36a24365a40f0737ac9f5e51]) successfully deleted
I20260620 09:25:29.634305 23512 catalog_manager.cc:4841] TS 9d365669c36f4e3d8d8a4f82cc5c5200 (127.4.59.2:41145): tablet 56025e93f7184833a10c8e58a67b9055 (table test-table [id=bc85398b36a24365a40f0737ac9f5e51]) successfully deleted
W20260620 09:25:29.635061 23515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.59.1:38223 (error 108)
I20260620 09:25:29.635303 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 23724
I20260620 09:25:29.635881 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.635405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:38223 (local address 127.0.0.1:56946)
0620 09:25:29.635806 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:29.635842 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382}
W20260620 09:25:29.636790 23515 catalog_manager.cc:4584] TS 142b019a37fe46989a156ebbb483eaf7 (127.4.59.1:38223): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 56025e93f7184833a10c8e58a67b9055: Network error: Client connection negotiation failed: client connection to 127.4.59.1:38223: connect: Connection refused (error 111)
I20260620 09:25:29.639966 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 23858
I20260620 09:25:29.644220 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 23496
2026-06-20T09:25:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1933 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-20T09:25:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:29Z Disabled control of system clock
I20260620 09:25:29.670436 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:42491
--webserver_interface=127.4.59.62
--webserver_port=0
--builtin_ntp_servers=127.4.59.20:39555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:42491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 09:25:29.749825 24031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:29.750051 24031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:29.750072 24031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:29.751137 24031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:29.751173 24031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:29.751185 24031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:29.751197 24031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:29.751209 24031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:29.752455 24031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:39555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:42491
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:42491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:29.752638 24031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:29.752859 24031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:29.757449 24036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:29.757444 24037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:29.757443 24039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:29.757696 24031 server_base.cc:1034] running on GCE node
I20260620 09:25:29.757995 24031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:29.758283 24031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:29.759461 24031 hybrid_clock.cc:648] HybridClock initialized: now 1781947529759444 us; error 27 us; skew 500 ppm
I20260620 09:25:29.760673 24031 webserver.cc:460] Webserver started at http://127.4.59.62:38157/ using document root <none> and password file <none>
I20260620 09:25:29.760926 24031 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:29.760998 24031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:29.761191 24031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:29.762219 24031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "5c7a2898dbdf4d588a17971d06424307"
format_stamp: "Formatted at 2026-06-20 09:25:29 on dist-test-slave-414w"
server_key: "360c0b32766be05c9f40d2dce52890a7"
server_key_iv: "0de39d25462e2a104f78043283b6c31d"
server_key_version: "encryptionkey@0"
I20260620 09:25:29.762581 24031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "5c7a2898dbdf4d588a17971d06424307"
format_stamp: "Formatted at 2026-06-20 09:25:29 on dist-test-slave-414w"
server_key: "360c0b32766be05c9f40d2dce52890a7"
server_key_iv: "0de39d25462e2a104f78043283b6c31d"
server_key_version: "encryptionkey@0"
I20260620 09:25:29.763921 24031 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 09:25:29.764693 24045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:29.764933 24031 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:25:29.765013 24031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "5c7a2898dbdf4d588a17971d06424307"
format_stamp: "Formatted at 2026-06-20 09:25:29 on dist-test-slave-414w"
server_key: "360c0b32766be05c9f40d2dce52890a7"
server_key_iv: "0de39d25462e2a104f78043283b6c31d"
server_key_version: "encryptionkey@0"
I20260620 09:25:29.765108 24031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:29.796911 24031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:29.797313 24031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:29.797519 24031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:29.801754 24031 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:42491
I20260620 09:25:29.801782 24097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:42491 every 8 connection(s)
I20260620 09:25:29.802151 24031 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:29.803066 24098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:29.804487 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 24031
I20260620 09:25:29.804600 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:29.804806 4332 external_mini_cluster.cc:1428] Setting key 1c2621185c41ca76b56af8f6cf02ba8d
I20260620 09:25:29.805712 24098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307: Bootstrap starting.
I20260620 09:25:29.806285 24098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:29.806591 24098 log.cc:824] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:29.807271 24098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307: No bootstrap required, opened a new log
I20260620 09:25:29.809303 24098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7a2898dbdf4d588a17971d06424307" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42491 } }
I20260620 09:25:29.809477 24098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:29.809523 24098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7a2898dbdf4d588a17971d06424307, State: Initialized, Role: FOLLOWER
I20260620 09:25:29.809676 24098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7a2898dbdf4d588a17971d06424307" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42491 } }
I20260620 09:25:29.809757 24098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:29.809779 24098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:29.809849 24098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:29.810415 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.805383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43848 (local address 127.4.59.62:42491)
0620 09:25:29.805583 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:29.805590 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:29.805614 (+ 24us) server_negotiation.cc:407] Connection header received
0620 09:25:29.806093 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:29.806100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:29.806161 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:29.806306 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:29.806544 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.807467 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.808408 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.808807 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.808941 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.809733 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:29.809736 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:29.809737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:29.809960 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:29.810015 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:29.810030 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:29.810084 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:29.810158 (+ 74us) server_negotiation.cc:299] Negotiation successful
0620 09:25:29.810234 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260620 09:25:29.810671 24098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7a2898dbdf4d588a17971d06424307" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42491 } }
I20260620 09:25:29.811007 24098 leader_election.cc:302] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c7a2898dbdf4d588a17971d06424307; no voters:
I20260620 09:25:29.811183 24098 leader_election.cc:288] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:29.811241 24103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:29.811381 24103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [term 1 LEADER]: Becoming Leader. State: Replica: 5c7a2898dbdf4d588a17971d06424307, State: Running, Role: LEADER
I20260620 09:25:29.811450 24098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:29.811466 24103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7a2898dbdf4d588a17971d06424307" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42491 } }
I20260620 09:25:29.811816 24104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c7a2898dbdf4d588a17971d06424307" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7a2898dbdf4d588a17971d06424307" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42491 } } }
I20260620 09:25:29.811817 24105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c7a2898dbdf4d588a17971d06424307. Latest consensus state: current_term: 1 leader_uuid: "5c7a2898dbdf4d588a17971d06424307" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7a2898dbdf4d588a17971d06424307" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 42491 } } }
I20260620 09:25:29.811918 24104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:29.811926 24105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:29.812147 24110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:29.812628 24110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:29.814222 24110 catalog_manager.cc:1347] Generated new cluster ID: fe8fa7cae4d8411a8af0eb60d0a7e25b
I20260620 09:25:29.814282 24110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:29.827095 24110 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:29.827833 24110 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:29.837261 24110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307: Generated new TSK 0
I20260620 09:25:29.837459 24110 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:29.839752 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.1:0
--local_ip_for_outbound_sockets=127.4.59.1
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:42491
--builtin_ntp_servers=127.4.59.20:39555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 09:25:29.931377 24122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:29.931569 24122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:29.931591 24122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:29.932715 24122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:29.932753 24122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:29.932780 24122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.1
I20260620 09:25:29.934340 24122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:39555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.59.1
--webserver_port=0
--tserver_master_addrs=127.4.59.62:42491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.1
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:29.934577 24122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:29.934926 24122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:29.941531 24127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:29.941514 24130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:29.941928 24128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:29.942972 24122 server_base.cc:1034] running on GCE node
I20260620 09:25:29.943192 24122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:29.943499 24122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:29.944674 24122 hybrid_clock.cc:648] HybridClock initialized: now 1781947529944657 us; error 30 us; skew 500 ppm
I20260620 09:25:29.946187 24122 webserver.cc:460] Webserver started at http://127.4.59.1:35451/ using document root <none> and password file <none>
I20260620 09:25:29.946468 24122 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:29.946545 24122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:29.946709 24122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:29.947808 24122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/data/instance:
uuid: "ead9cc0dc47342ec85010f59303d4a3d"
format_stamp: "Formatted at 2026-06-20 09:25:29 on dist-test-slave-414w"
server_key: "9115444d483ab1d3e711db73a5b76533"
server_key_iv: "0e8e93ad96af9232af895de606518617"
server_key_version: "encryptionkey@0"
I20260620 09:25:29.948186 24122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance:
uuid: "ead9cc0dc47342ec85010f59303d4a3d"
format_stamp: "Formatted at 2026-06-20 09:25:29 on dist-test-slave-414w"
server_key: "9115444d483ab1d3e711db73a5b76533"
server_key_iv: "0e8e93ad96af9232af895de606518617"
server_key_version: "encryptionkey@0"
I20260620 09:25:29.949618 24122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:25:29.950412 24136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:29.950558 24122 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:29.950662 24122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/wal
uuid: "ead9cc0dc47342ec85010f59303d4a3d"
format_stamp: "Formatted at 2026-06-20 09:25:29 on dist-test-slave-414w"
server_key: "9115444d483ab1d3e711db73a5b76533"
server_key_iv: "0e8e93ad96af9232af895de606518617"
server_key_version: "encryptionkey@0"
I20260620 09:25:29.950734 24122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:29.963934 24122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:29.964213 24122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:29.964360 24122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:29.964574 24122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:29.964923 24122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:29.964959 24122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:29.964983 24122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:29.965026 24122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:29.970672 24122 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.1:43665
I20260620 09:25:29.970784 24249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.1:43665 every 8 connection(s)
I20260620 09:25:29.971060 24122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/data/info.pb
I20260620 09:25:29.975098 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 24122
I20260620 09:25:29.975188 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-0/wal/instance
I20260620 09:25:29.975392 4332 external_mini_cluster.cc:1428] Setting key bb3f6e6762109bf9cd3bf1598f9d4f19
I20260620 09:25:29.976147 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.971713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.1:59455 (local address 127.4.59.62:42491)
0620 09:25:29.971782 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:29.971784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:29.972001 (+ 217us) server_negotiation.cc:407] Connection header received
0620 09:25:29.972447 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:29.972448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:29.972458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:29.972511 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:29.973180 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.973689 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.974620 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.974819 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.974885 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.975840 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:29.975841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:29.975841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:29.975845 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:29.975858 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:29.975862 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:29.975908 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:29.976018 (+ 110us) server_negotiation.cc:299] Negotiation successful
0620 09:25:29.976044 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 09:25:29.976331 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:29.971717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42491 (local address 127.4.59.1:59455)
0620 09:25:29.971878 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:29.971909 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:29.972366 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:29.972581 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:29.972584 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:29.972591 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:29.973063 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.973067 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.973765 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.973765 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.974524 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:29.974526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:29.974901 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:29.974902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:29.975019 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:29.975680 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:25:29.975681 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:29.975717 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:25:29.975741 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:29.975924 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:29.975929 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:29.975932 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:25:29.976000 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 09:25:29.976102 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
I20260620 09:25:29.976460 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.2:0
--local_ip_for_outbound_sockets=127.4.59.2
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:42491
--builtin_ntp_servers=127.4.59.20:39555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 09:25:29.976735 24250 heartbeater.cc:347] Connected to a master server at 127.4.59.62:42491
I20260620 09:25:29.976846 24250 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:29.977020 24250 heartbeater.cc:511] Master 127.4.59.62:42491 requested a full tablet report, sending...
I20260620 09:25:29.977478 24062 ts_manager.cc:194] Registered new tserver with Master: ead9cc0dc47342ec85010f59303d4a3d (127.4.59.1:43665)
I20260620 09:25:29.978084 24062 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.59.1:59455
W20260620 09:25:30.055851 24253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:30.056038 24253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:30.056059 24253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:30.057384 24253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:30.057423 24253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:30.057449 24253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.2
I20260620 09:25:30.058753 24253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:39555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.59.2
--webserver_port=0
--tserver_master_addrs=127.4.59.62:42491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.2
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:30.058971 24253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:30.059268 24253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:30.064240 24259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:30.064307 24258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:30.064249 24261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:30.064683 24253 server_base.cc:1034] running on GCE node
I20260620 09:25:30.064924 24253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:30.065196 24253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:30.066394 24253 hybrid_clock.cc:648] HybridClock initialized: now 1781947530066378 us; error 34 us; skew 500 ppm
I20260620 09:25:30.067795 24253 webserver.cc:460] Webserver started at http://127.4.59.2:41573/ using document root <none> and password file <none>
I20260620 09:25:30.068266 24253 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:30.068372 24253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:30.068508 24253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:30.069723 24253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/data/instance:
uuid: "9c7260bffb5243b0b0ad104742f3528b"
format_stamp: "Formatted at 2026-06-20 09:25:30 on dist-test-slave-414w"
server_key: "93088584ad31823a2d2dd40b278d2ada"
server_key_iv: "4251a80ac05935a754eef049c1d92ce2"
server_key_version: "encryptionkey@0"
I20260620 09:25:30.070072 24253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance:
uuid: "9c7260bffb5243b0b0ad104742f3528b"
format_stamp: "Formatted at 2026-06-20 09:25:30 on dist-test-slave-414w"
server_key: "93088584ad31823a2d2dd40b278d2ada"
server_key_iv: "4251a80ac05935a754eef049c1d92ce2"
server_key_version: "encryptionkey@0"
I20260620 09:25:30.071311 24253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:25:30.072027 24267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:30.072191 24253 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:30.072285 24253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/wal
uuid: "9c7260bffb5243b0b0ad104742f3528b"
format_stamp: "Formatted at 2026-06-20 09:25:30 on dist-test-slave-414w"
server_key: "93088584ad31823a2d2dd40b278d2ada"
server_key_iv: "4251a80ac05935a754eef049c1d92ce2"
server_key_version: "encryptionkey@0"
I20260620 09:25:30.072366 24253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:30.080188 24253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:30.080443 24253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:30.080582 24253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:30.080781 24253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:30.081103 24253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:30.081195 24253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:30.081238 24253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:30.081267 24253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:30.086805 24253 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.2:34505
I20260620 09:25:30.086848 24380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.2:34505 every 8 connection(s)
I20260620 09:25:30.087136 24253 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/data/info.pb
I20260620 09:25:30.091090 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 24253
I20260620 09:25:30.091212 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/wal/instance
I20260620 09:25:30.091408 4332 external_mini_cluster.cc:1428] Setting key b922afae871ba8100707fe210da700f0
I20260620 09:25:30.092249 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.087787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.2:43833 (local address 127.4.59.62:42491)
0620 09:25:30.087852 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:30.087854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:30.088057 (+ 203us) server_negotiation.cc:407] Connection header received
0620 09:25:30.088548 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:30.088549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:30.088561 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:30.088626 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:30.089303 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.089807 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.090774 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.090982 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.091066 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:30.091972 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:30.091973 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:30.091974 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:30.091979 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:30.091995 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:30.091999 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:30.092073 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:30.092138 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 09:25:30.092168 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 09:25:30.092437 24383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.087761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42491 (local address 127.4.59.2:43833)
0620 09:25:30.087949 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:30.087980 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:30.088448 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:30.088699 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:30.088702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:30.088710 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:30.089214 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:30.089219 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.089908 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.089909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:30.090674 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:30.090676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.091075 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.091076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:30.091229 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:30.091844 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:25:30.091845 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:30.091871 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:25:30.091892 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:30.092066 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:30.092072 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:30.092074 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 09:25:30.092139 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 09:25:30.092224 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":82,"threads_started":1}
I20260620 09:25:30.092450 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.59.3:0
--local_ip_for_outbound_sockets=127.4.59.3
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:42491
--builtin_ntp_servers=127.4.59.20:39555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 09:25:30.092940 24381 heartbeater.cc:347] Connected to a master server at 127.4.59.62:42491
I20260620 09:25:30.093037 24381 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:30.093281 24381 heartbeater.cc:511] Master 127.4.59.62:42491 requested a full tablet report, sending...
I20260620 09:25:30.093654 24062 ts_manager.cc:194] Registered new tserver with Master: 9c7260bffb5243b0b0ad104742f3528b (127.4.59.2:34505)
I20260620 09:25:30.094033 24062 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.59.2:43833
W20260620 09:25:30.172714 24384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:30.172916 24384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:30.172940 24384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:30.174124 24384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:30.174165 24384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:30.174189 24384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.59.3
I20260620 09:25:30.175594 24384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:39555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.59.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.59.3
--webserver_port=0
--tserver_master_addrs=127.4.59.62:42491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.59.3
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:30.175855 24384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:30.176149 24384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:30.181192 24390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:30.181222 24392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:30.181253 24384 server_base.cc:1034] running on GCE node
W20260620 09:25:30.181209 24389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:30.181668 24384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:30.181910 24384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:30.183080 24384 hybrid_clock.cc:648] HybridClock initialized: now 1781947530183076 us; error 34 us; skew 500 ppm
I20260620 09:25:30.184337 24384 webserver.cc:460] Webserver started at http://127.4.59.3:34749/ using document root <none> and password file <none>
I20260620 09:25:30.184587 24384 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:30.184657 24384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:30.184815 24384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:30.185886 24384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/data/instance:
uuid: "cc5532b75b0d4a5b8aeb5264021b44c3"
format_stamp: "Formatted at 2026-06-20 09:25:30 on dist-test-slave-414w"
server_key: "02c3c3941a3dffe53bc3b6db7915dd75"
server_key_iv: "cb3d6173b015bec6fb2ce9558fad6114"
server_key_version: "encryptionkey@0"
I20260620 09:25:30.186235 24384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance:
uuid: "cc5532b75b0d4a5b8aeb5264021b44c3"
format_stamp: "Formatted at 2026-06-20 09:25:30 on dist-test-slave-414w"
server_key: "02c3c3941a3dffe53bc3b6db7915dd75"
server_key_iv: "cb3d6173b015bec6fb2ce9558fad6114"
server_key_version: "encryptionkey@0"
I20260620 09:25:30.187462 24384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 09:25:30.188154 24398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:30.188331 24384 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:25:30.188424 24384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/wal
uuid: "cc5532b75b0d4a5b8aeb5264021b44c3"
format_stamp: "Formatted at 2026-06-20 09:25:30 on dist-test-slave-414w"
server_key: "02c3c3941a3dffe53bc3b6db7915dd75"
server_key_iv: "cb3d6173b015bec6fb2ce9558fad6114"
server_key_version: "encryptionkey@0"
I20260620 09:25:30.188503 24384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:30.196893 24384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:30.197207 24384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:30.197356 24384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:30.197556 24384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 09:25:30.197852 24384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 09:25:30.197906 24384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:30.197944 24384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 09:25:30.197970 24384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:30.203408 24384 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.3:43875
I20260620 09:25:30.203488 24511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.3:43875 every 8 connection(s)
I20260620 09:25:30.203778 24384 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/data/info.pb
I20260620 09:25:30.207496 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 24384
I20260620 09:25:30.207594 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/wal/instance
I20260620 09:25:30.207878 4332 external_mini_cluster.cc:1428] Setting key 28e9e9be3017d5cf11e99cf1533ff75f
I20260620 09:25:30.208897 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.204462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:57223 (local address 127.4.59.62:42491)
0620 09:25:30.204537 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:30.204540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:30.204732 (+ 192us) server_negotiation.cc:407] Connection header received
0620 09:25:30.205236 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:30.205238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:30.205250 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:30.205315 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:30.205943 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.206419 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.207344 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.207656 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.207734 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:30.208632 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:30.208633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:30.208633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:30.208636 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:30.208650 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:30.208653 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:30.208724 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:30.208800 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 09:25:30.208824 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 09:25:30.209086 24514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.204423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:42491 (local address 127.4.59.3:57223)
0620 09:25:30.204614 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:30.204645 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:30.205085 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:30.205386 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:30.205390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:30.205397 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:30.205860 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:30.205864 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.206495 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.206495 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:30.207238 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:30.207239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.207746 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.207747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:30.207861 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:30.208478 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 09:25:30.208478 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:30.208504 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 09:25:30.208548 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:30.208727 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:30.208731 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:30.208734 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:25:30.208793 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 09:25:30.208871 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
I20260620 09:25:30.209538 24512 heartbeater.cc:347] Connected to a master server at 127.4.59.62:42491
I20260620 09:25:30.209609 24512 heartbeater.cc:464] Registering TS with master...
I20260620 09:25:30.209797 24512 heartbeater.cc:511] Master 127.4.59.62:42491 requested a full tablet report, sending...
I20260620 09:25:30.210201 24062 ts_manager.cc:194] Registered new tserver with Master: cc5532b75b0d4a5b8aeb5264021b44c3 (127.4.59.3:43875)
I20260620 09:25:30.210552 24062 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.59.3:57223
I20260620 09:25:30.219614 4332 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 09:25:30.223858 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.221006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43858 (local address 127.4.59.62:42491)
0620 09:25:30.221093 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:30.221095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:30.221211 (+ 116us) server_negotiation.cc:407] Connection header received
0620 09:25:30.221284 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:30.221285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:30.221295 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:30.221383 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:30.221591 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.222056 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.222776 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.222949 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.223006 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:30.223579 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:30.223580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:30.223580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:30.223584 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:30.223593 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:30.223596 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:30.223639 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:30.223741 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 09:25:30.223757 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 09:25:30.225477 24062 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43858:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 09:25:30.227025 24062 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 09:25:30.233088 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.229324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:43665 (local address 127.0.0.1:46166)
0620 09:25:30.229678 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:30.229686 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:30.229754 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:30.229917 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:30.229918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:30.229920 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:30.229974 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:30.229976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.230885 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.230886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:30.231972 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:30.231975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.232814 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.232815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:30.232884 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:30.232896 (+ 12us) client_negotiation.cc:769] Sending connection context
0620 09:25:30.232970 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 09:25:30.233018 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":64,"threads_started":1}
I20260620 09:25:30.233107 24525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.229417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46166 (local address 127.4.59.1:43665)
0620 09:25:30.229614 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:30.229618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:30.229750 (+ 132us) server_negotiation.cc:407] Connection header received
0620 09:25:30.229821 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:30.229823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:30.229839 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:30.229918 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:30.230054 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.230787 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.232068 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.232729 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.232814 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:30.232904 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:30.232983 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 09:25:30.233019 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":96,"threads_started":1}
I20260620 09:25:30.233491 24520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.229173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:34505 (local address 127.0.0.1:55050)
0620 09:25:30.229440 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:30.229472 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:30.229531 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:30.229773 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:30.229776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:30.229778 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:30.230114 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:30.230117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.231375 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.231376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:30.232230 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:30.232232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.233241 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.233242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:30.233300 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:30.233303 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:25:30.233371 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 09:25:30.233407 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":8,"thread_start_us":113,"threads_started":1}
I20260620 09:25:30.233543 24524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.229427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55050 (local address 127.4.59.2:34505)
0620 09:25:30.229613 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:30.229617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:30.229626 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:25:30.229663 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:30.229665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:30.229683 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:30.229770 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:30.230200 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.231293 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.232314 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.233029 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.233098 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:30.233205 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:30.233404 (+ 199us) server_negotiation.cc:299] Negotiation successful
0620 09:25:30.233449 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
I20260620 09:25:30.233680 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.229204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50946 (local address 127.4.59.3:43875)
0620 09:25:30.229877 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:30.229882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:30.229891 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:25:30.229923 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:30.229925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:30.229939 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:30.230014 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:30.230114 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.231242 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.232479 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.233187 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.233247 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:30.233322 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:30.233530 (+ 208us) server_negotiation.cc:299] Negotiation successful
0620 09:25:30.233577 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":68,"threads_started":1}
I20260620 09:25:30.233865 24521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.229089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.3:43875 (local address 127.0.0.1:50946)
0620 09:25:30.229430 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:30.229470 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:30.229532 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:30.230012 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:30.230013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:30.230015 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:30.230055 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:30.230056 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.231336 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.231337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:30.232401 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:30.232403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.233353 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.233354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:30.233408 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:30.233411 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:25:30.233455 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 09:25:30.233484 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":84,"threads_started":1}
I20260620 09:25:30.234241 24184 tablet_service.cc:1467] Processing CreateTablet for tablet cc468b312ce14724bf6d79376d1e756f (DEFAULT_TABLE table=test-table [id=f1dba7e8e40e421da05dd55526cb9059]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:30.234416 24446 tablet_service.cc:1467] Processing CreateTablet for tablet cc468b312ce14724bf6d79376d1e756f (DEFAULT_TABLE table=test-table [id=f1dba7e8e40e421da05dd55526cb9059]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:30.234622 24184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc468b312ce14724bf6d79376d1e756f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:30.234683 24446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc468b312ce14724bf6d79376d1e756f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:30.237914 24526 tablet_bootstrap.cc:492] T cc468b312ce14724bf6d79376d1e756f P ead9cc0dc47342ec85010f59303d4a3d: Bootstrap starting.
I20260620 09:25:30.238061 24527 tablet_bootstrap.cc:492] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3: Bootstrap starting.
I20260620 09:25:30.238210 24315 tablet_service.cc:1467] Processing CreateTablet for tablet cc468b312ce14724bf6d79376d1e756f (DEFAULT_TABLE table=test-table [id=f1dba7e8e40e421da05dd55526cb9059]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 09:25:30.238530 24315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc468b312ce14724bf6d79376d1e756f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:30.238658 24527 tablet_bootstrap.cc:654] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:30.238658 24526 tablet_bootstrap.cc:654] T cc468b312ce14724bf6d79376d1e756f P ead9cc0dc47342ec85010f59303d4a3d: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:30.238991 24526 log.cc:824] T cc468b312ce14724bf6d79376d1e756f P ead9cc0dc47342ec85010f59303d4a3d: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:30.238991 24527 log.cc:824] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:30.239703 24526 tablet_bootstrap.cc:492] T cc468b312ce14724bf6d79376d1e756f P ead9cc0dc47342ec85010f59303d4a3d: No bootstrap required, opened a new log
I20260620 09:25:30.239776 24527 tablet_bootstrap.cc:492] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3: No bootstrap required, opened a new log
I20260620 09:25:30.239853 24527 ts_tablet_manager.cc:1397] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:25:30.239876 24526 ts_tablet_manager.cc:1397] T cc468b312ce14724bf6d79376d1e756f P ead9cc0dc47342ec85010f59303d4a3d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 09:25:30.241606 24527 raft_consensus.cc:348] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5532b75b0d4a5b8aeb5264021b44c3" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43875 } } peers { permanent_uuid: "ead9cc0dc47342ec85010f59303d4a3d" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 43665 } } peers { permanent_uuid: "9c7260bffb5243b0b0ad104742f3528b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34505 } }
I20260620 09:25:30.241812 24527 raft_consensus.cc:374] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:30.241865 24527 raft_consensus.cc:729] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc5532b75b0d4a5b8aeb5264021b44c3, State: Initialized, Role: FOLLOWER
I20260620 09:25:30.241869 24530 tablet_bootstrap.cc:492] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b: Bootstrap starting.
I20260620 09:25:30.241987 24527 consensus_queue.cc:260] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5532b75b0d4a5b8aeb5264021b44c3" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43875 } } peers { permanent_uuid: "ead9cc0dc47342ec85010f59303d4a3d" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 43665 } } peers { permanent_uuid: "9c7260bffb5243b0b0ad104742f3528b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34505 } }
I20260620 09:25:30.242271 24527 ts_tablet_manager.cc:1428] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 09:25:30.242375 24512 heartbeater.cc:503] Master 127.4.59.62:42491 was elected leader, sending a full tablet report...
I20260620 09:25:30.242712 24530 tablet_bootstrap.cc:654] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:30.242666 24526 raft_consensus.cc:348] T cc468b312ce14724bf6d79376d1e756f P ead9cc0dc47342ec85010f59303d4a3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5532b75b0d4a5b8aeb5264021b44c3" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43875 } } peers { permanent_uuid: "ead9cc0dc47342ec85010f59303d4a3d" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 43665 } } peers { permanent_uuid: "9c7260bffb5243b0b0ad104742f3528b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34505 } }
I20260620 09:25:30.242812 24526 raft_consensus.cc:374] T cc468b312ce14724bf6d79376d1e756f P ead9cc0dc47342ec85010f59303d4a3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:30.242871 24526 raft_consensus.cc:729] T cc468b312ce14724bf6d79376d1e756f P ead9cc0dc47342ec85010f59303d4a3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead9cc0dc47342ec85010f59303d4a3d, State: Initialized, Role: FOLLOWER
I20260620 09:25:30.242980 24526 consensus_queue.cc:260] T cc468b312ce14724bf6d79376d1e756f P ead9cc0dc47342ec85010f59303d4a3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5532b75b0d4a5b8aeb5264021b44c3" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43875 } } peers { permanent_uuid: "ead9cc0dc47342ec85010f59303d4a3d" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 43665 } } peers { permanent_uuid: "9c7260bffb5243b0b0ad104742f3528b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34505 } }
I20260620 09:25:30.243026 24530 log.cc:824] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:30.243248 24526 ts_tablet_manager.cc:1428] T cc468b312ce14724bf6d79376d1e756f P ead9cc0dc47342ec85010f59303d4a3d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 09:25:30.243273 24250 heartbeater.cc:503] Master 127.4.59.62:42491 was elected leader, sending a full tablet report...
I20260620 09:25:30.243822 24530 tablet_bootstrap.cc:492] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b: No bootstrap required, opened a new log
I20260620 09:25:30.243887 24530 ts_tablet_manager.cc:1397] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 09:25:30.245782 24530 raft_consensus.cc:348] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5532b75b0d4a5b8aeb5264021b44c3" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43875 } } peers { permanent_uuid: "ead9cc0dc47342ec85010f59303d4a3d" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 43665 } } peers { permanent_uuid: "9c7260bffb5243b0b0ad104742f3528b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34505 } }
I20260620 09:25:30.245931 24530 raft_consensus.cc:374] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:30.245955 24530 raft_consensus.cc:729] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c7260bffb5243b0b0ad104742f3528b, State: Initialized, Role: FOLLOWER
I20260620 09:25:30.246052 24530 consensus_queue.cc:260] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5532b75b0d4a5b8aeb5264021b44c3" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43875 } } peers { permanent_uuid: "ead9cc0dc47342ec85010f59303d4a3d" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 43665 } } peers { permanent_uuid: "9c7260bffb5243b0b0ad104742f3528b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34505 } }
I20260620 09:25:30.246287 24530 ts_tablet_manager.cc:1428] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 09:25:30.246321 24381 heartbeater.cc:503] Master 127.4.59.62:42491 was elected leader, sending a full tablet report...
W20260620 09:25:30.337630 24382 tablet.cc:2369] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 09:25:30.454334 24513 tablet.cc:2369] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:25:30.459518 24531 raft_consensus.cc:482] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 09:25:30.459601 24531 raft_consensus.cc:504] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5532b75b0d4a5b8aeb5264021b44c3" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43875 } } peers { permanent_uuid: "ead9cc0dc47342ec85010f59303d4a3d" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 43665 } } peers { permanent_uuid: "9c7260bffb5243b0b0ad104742f3528b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34505 } }
I20260620 09:25:30.459889 24531 leader_election.cc:288] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ead9cc0dc47342ec85010f59303d4a3d (127.4.59.1:43665), 9c7260bffb5243b0b0ad104742f3528b (127.4.59.2:34505)
I20260620 09:25:30.462846 24514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.460059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:43665 (local address 127.4.59.3:39629)
0620 09:25:30.460119 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:30.460134 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:30.460186 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:30.460354 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:30.460355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:30.460356 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:30.460423 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:30.460425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.461365 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.461366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:30.462084 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:30.462086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.462576 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.462577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:30.462654 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:30.462657 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 09:25:30.462731 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 09:25:30.462775 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 09:25:30.462846 24525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.460188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:39629 (local address 127.4.59.1:43665)
0620 09:25:30.460232 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:30.460234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:30.460242 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:25:30.460267 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:30.460267 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:30.460285 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:30.460348 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:30.460493 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.461288 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.462157 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.462491 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.462575 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:30.462584 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:30.462739 (+ 155us) server_negotiation.cc:299] Negotiation successful
0620 09:25:30.462776 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 09:25:30.463160 24204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc468b312ce14724bf6d79376d1e756f" candidate_uuid: "cc5532b75b0d4a5b8aeb5264021b44c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead9cc0dc47342ec85010f59303d4a3d" is_pre_election: true
I20260620 09:25:30.463333 24204 raft_consensus.cc:2454] T cc468b312ce14724bf6d79376d1e756f P ead9cc0dc47342ec85010f59303d4a3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc5532b75b0d4a5b8aeb5264021b44c3 in term 0.
I20260620 09:25:30.463546 24402 leader_election.cc:302] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc5532b75b0d4a5b8aeb5264021b44c3, ead9cc0dc47342ec85010f59303d4a3d; no voters:
I20260620 09:25:30.463681 24531 raft_consensus.cc:2781] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 09:25:30.463745 24531 raft_consensus.cc:482] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 09:25:30.463776 24531 raft_consensus.cc:3037] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:30.463858 24524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.460189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.59.3:33809 (local address 127.4.59.2:34505)
0620 09:25:30.460235 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:30.460237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:30.460292 (+ 55us) server_negotiation.cc:407] Connection header received
0620 09:25:30.460373 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:30.460374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:30.460383 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:30.460426 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 09:25:30.460587 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.461303 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.462927 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.463498 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.463594 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:30.463605 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:30.463749 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 09:25:30.463798 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 09:25:30.463860 24535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.460073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.2:34505 (local address 127.4.59.3:33809)
0620 09:25:30.460230 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:30.460241 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:30.460317 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:30.460447 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:30.460448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:30.460450 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 09:25:30.460516 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:30.460518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.462165 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.462165 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:30.462841 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:30.462843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.463608 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.463609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:30.463681 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:30.463685 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 09:25:30.463742 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 09:25:30.463777 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":12,"thread_start_us":88,"threads_started":1}
I20260620 09:25:30.464128 24335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc468b312ce14724bf6d79376d1e756f" candidate_uuid: "cc5532b75b0d4a5b8aeb5264021b44c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7260bffb5243b0b0ad104742f3528b" is_pre_election: true
I20260620 09:25:30.464229 24335 raft_consensus.cc:2454] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc5532b75b0d4a5b8aeb5264021b44c3 in term 0.
I20260620 09:25:30.464622 24531 raft_consensus.cc:504] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5532b75b0d4a5b8aeb5264021b44c3" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43875 } } peers { permanent_uuid: "ead9cc0dc47342ec85010f59303d4a3d" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 43665 } } peers { permanent_uuid: "9c7260bffb5243b0b0ad104742f3528b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34505 } }
I20260620 09:25:30.464764 24531 leader_election.cc:288] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [CANDIDATE]: Term 1 election: Requested vote from peers ead9cc0dc47342ec85010f59303d4a3d (127.4.59.1:43665), 9c7260bffb5243b0b0ad104742f3528b (127.4.59.2:34505)
I20260620 09:25:30.464906 24335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc468b312ce14724bf6d79376d1e756f" candidate_uuid: "cc5532b75b0d4a5b8aeb5264021b44c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7260bffb5243b0b0ad104742f3528b"
I20260620 09:25:30.464922 24204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc468b312ce14724bf6d79376d1e756f" candidate_uuid: "cc5532b75b0d4a5b8aeb5264021b44c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead9cc0dc47342ec85010f59303d4a3d"
I20260620 09:25:30.464982 24335 raft_consensus.cc:3037] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:30.465008 24204 raft_consensus.cc:3037] T cc468b312ce14724bf6d79376d1e756f P ead9cc0dc47342ec85010f59303d4a3d [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:30.465687 24204 raft_consensus.cc:2454] T cc468b312ce14724bf6d79376d1e756f P ead9cc0dc47342ec85010f59303d4a3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc5532b75b0d4a5b8aeb5264021b44c3 in term 1.
I20260620 09:25:30.465736 24335 raft_consensus.cc:2454] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc5532b75b0d4a5b8aeb5264021b44c3 in term 1.
I20260620 09:25:30.465870 24402 leader_election.cc:302] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc5532b75b0d4a5b8aeb5264021b44c3, ead9cc0dc47342ec85010f59303d4a3d; no voters:
I20260620 09:25:30.465946 24531 raft_consensus.cc:2781] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:30.466068 24531 raft_consensus.cc:686] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [term 1 LEADER]: Becoming Leader. State: Replica: cc5532b75b0d4a5b8aeb5264021b44c3, State: Running, Role: LEADER
I20260620 09:25:30.466159 24531 consensus_queue.cc:237] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5532b75b0d4a5b8aeb5264021b44c3" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43875 } } peers { permanent_uuid: "ead9cc0dc47342ec85010f59303d4a3d" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 43665 } } peers { permanent_uuid: "9c7260bffb5243b0b0ad104742f3528b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34505 } }
I20260620 09:25:30.466959 24062 catalog_manager.cc:5495] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc5532b75b0d4a5b8aeb5264021b44c3 (127.4.59.3). New cstate: current_term: 1 leader_uuid: "cc5532b75b0d4a5b8aeb5264021b44c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5532b75b0d4a5b8aeb5264021b44c3" member_type: VOTER last_known_addr { host: "127.4.59.3" port: 43875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ead9cc0dc47342ec85010f59303d4a3d" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 43665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7260bffb5243b0b0ad104742f3528b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34505 } health_report { overall_health: UNKNOWN } } }
W20260620 09:25:30.471701 24251 tablet.cc:2369] T cc468b312ce14724bf6d79376d1e756f P ead9cc0dc47342ec85010f59303d4a3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 09:25:30.519470 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.516940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50948 (local address 127.4.59.3:43875)
0620 09:25:30.517004 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:30.517006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:30.517016 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:25:30.517090 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:30.517091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:30.517108 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:30.517227 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:25:30.517380 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.517862 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.518641 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:30.518852 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:30.518931 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:30.519109 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:25:30.519206 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:25:30.519296 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:30.519337 (+ 41us) server_negotiation.cc:299] Negotiation successful
0620 09:25:30.519356 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 09:25:30.521384 24335 raft_consensus.cc:1261] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b [term 1 FOLLOWER]: Refusing update from remote peer cc5532b75b0d4a5b8aeb5264021b44c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:25:30.521384 24204 raft_consensus.cc:1261] T cc468b312ce14724bf6d79376d1e756f P ead9cc0dc47342ec85010f59303d4a3d [term 1 FOLLOWER]: Refusing update from remote peer cc5532b75b0d4a5b8aeb5264021b44c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:25:30.521795 24536 consensus_queue.cc:1035] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead9cc0dc47342ec85010f59303d4a3d" member_type: VOTER last_known_addr { host: "127.4.59.1" port: 43665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:25:30.521934 24531 consensus_queue.cc:1035] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7260bffb5243b0b0ad104742f3528b" member_type: VOTER last_known_addr { host: "127.4.59.2" port: 34505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:25:30.524703 24540 mvcc.cc:205] Tried to move back new op lower bound from 7298857085012905984 to 7298857084790177792. Current Snapshot: MvccSnapshot[applied={T|T < 7298857085012905984}]
I20260620 09:25:30.525339 24541 mvcc.cc:205] Tried to move back new op lower bound from 7298857085012905984 to 7298857084790177792. Current Snapshot: MvccSnapshot[applied={T|T < 7298857085012905984 or (T in {7298857085012905984})}]
I20260620 09:25:30.526659 24061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:25:30.526775 24061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 09:25:30.527562 24061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c7a2898dbdf4d588a17971d06424307: Sending DeleteTablet for 3 replicas of tablet cc468b312ce14724bf6d79376d1e756f
I20260620 09:25:30.527932 24184 tablet_service.cc:1514] Processing DeleteTablet for tablet cc468b312ce14724bf6d79376d1e756f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:30 UTC) from {username='slave'} at 127.0.0.1:46166
I20260620 09:25:30.527935 24446 tablet_service.cc:1514] Processing DeleteTablet for tablet cc468b312ce14724bf6d79376d1e756f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:30 UTC) from {username='slave'} at 127.0.0.1:50946
I20260620 09:25:30.528173 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 24122
I20260620 09:25:30.528374 24549 tablet_replica.cc:330] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3: stopping tablet replica
I20260620 09:25:30.528470 24549 raft_consensus.cc:2229] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [term 1 LEADER]: Raft consensus shutting down.
I20260620 09:25:30.528553 24549 raft_consensus.cc:2258] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:25:30.528964 24315 tablet_service.cc:1514] Processing DeleteTablet for tablet cc468b312ce14724bf6d79376d1e756f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 09:25:30 UTC) from {username='slave'} at 127.0.0.1:55050
I20260620 09:25:30.529202 24551 tablet_replica.cc:330] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b: stopping tablet replica
I20260620 09:25:30.529363 24551 raft_consensus.cc:2229] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 09:25:30.529511 24551 raft_consensus.cc:2258] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 09:25:30.530099 24551 ts_tablet_manager.cc:1905] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:25:30.530886 24549 ts_tablet_manager.cc:1905] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 09:25:30.532248 24551 ts_tablet_manager.cc:1918] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:25:30.532389 24551 log.cc:1196] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-1/wal/wals/cc468b312ce14724bf6d79376d1e756f
I20260620 09:25:30.532466 24549 ts_tablet_manager.cc:1918] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 09:25:30.532541 24549 log.cc:1196] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3: Deleting WAL directory at /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781947262065055-4332-0/minicluster-data/ts-2/wal/wals/cc468b312ce14724bf6d79376d1e756f
I20260620 09:25:30.532806 24551 ts_tablet_manager.cc:1939] T cc468b312ce14724bf6d79376d1e756f P 9c7260bffb5243b0b0ad104742f3528b: Deleting consensus metadata
I20260620 09:25:30.532833 24549 ts_tablet_manager.cc:1939] T cc468b312ce14724bf6d79376d1e756f P cc5532b75b0d4a5b8aeb5264021b44c3: Deleting consensus metadata
I20260620 09:25:30.533268 24049 catalog_manager.cc:4841] TS cc5532b75b0d4a5b8aeb5264021b44c3 (127.4.59.3:43875): tablet cc468b312ce14724bf6d79376d1e756f (table test-table [id=f1dba7e8e40e421da05dd55526cb9059]) successfully deleted
I20260620 09:25:30.533464 24046 catalog_manager.cc:4841] TS 9c7260bffb5243b0b0ad104742f3528b (127.4.59.2:34505): tablet cc468b312ce14724bf6d79376d1e756f (table test-table [id=f1dba7e8e40e421da05dd55526cb9059]) successfully deleted
W20260620 09:25:30.534900 24049 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.59.1:43665 (error 108)
I20260620 09:25:30.535421 24521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:30.535274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.1:43665 (local address 127.0.0.1:46176)
0620 09:25:30.535337 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:30.535383 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.1:43665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260620 09:25:30.535490 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 24253
W20260620 09:25:30.540637 24049 catalog_manager.cc:4584] TS ead9cc0dc47342ec85010f59303d4a3d (127.4.59.1:43665): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cc468b312ce14724bf6d79376d1e756f: Network error: Client connection negotiation failed: client connection to 127.4.59.1:43665: connect: Connection refused (error 111)
I20260620 09:25:30.540818 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 24384
I20260620 09:25:30.545756 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 24031
2026-06-20T09:25:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (901 ms)
[----------] 6 tests from EncryptionPolicyTest (8295 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:30 dist-test-slave-414w krb5kdc[24555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:30 dist-test-slave-414w krb5kdc[24555](info): set up 2 sockets
Jun 20 09:25:30 dist-test-slave-414w krb5kdc[24555](info): commencing operation
krb5kdc: starting...
W20260620 09:25:31.585412 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:31 dist-test-slave-414w krb5kdc[24555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 09:25:31.697553 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34225
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:31.777597 24568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:31.777797 24568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:31.777833 24568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:31.778934 24568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:31.779001 24568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:31.779028 24568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:31.779055 24568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:31.779091 24568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:31.780515 24568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34225
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:31.780787 24568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:31.781001 24568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:31.785686 24576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:31.785691 24574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:31.785730 24573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:31.785915 24568 server_base.cc:1034] running on GCE node
I20260620 09:25:31.786135 24568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:31.786378 24568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:31.787523 24568 hybrid_clock.cc:648] HybridClock initialized: now 1781947531787508 us; error 28 us; skew 500 ppm
Jun 20 09:25:31 dist-test-slave-414w krb5kdc[24555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947531, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:31.789848 24568 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 09:25:31.790293 24568 webserver.cc:460] Webserver started at http://127.0.0.1:45431/ using document root <none> and password file <none>
I20260620 09:25:31.790521 24568 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:31.790563 24568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:31.790714 24568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:31.791675 24568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "f72f36dca90e4a2fa3041647e5772ae6"
format_stamp: "Formatted at 2026-06-20 09:25:31 on dist-test-slave-414w"
server_key: "ba67d074dbd3f97029ee84110b8e2714"
server_key_iv: "0dcc442cbdd4b96eb9145f52d7f26f8c"
server_key_version: "encryptionkey@0"
I20260620 09:25:31.792054 24568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "f72f36dca90e4a2fa3041647e5772ae6"
format_stamp: "Formatted at 2026-06-20 09:25:31 on dist-test-slave-414w"
server_key: "ba67d074dbd3f97029ee84110b8e2714"
server_key_iv: "0dcc442cbdd4b96eb9145f52d7f26f8c"
server_key_version: "encryptionkey@0"
I20260620 09:25:31.794013 24568 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260620 09:25:31.795022 24583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:31.795233 24568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:31.795346 24568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "f72f36dca90e4a2fa3041647e5772ae6"
format_stamp: "Formatted at 2026-06-20 09:25:31 on dist-test-slave-414w"
server_key: "ba67d074dbd3f97029ee84110b8e2714"
server_key_iv: "0dcc442cbdd4b96eb9145f52d7f26f8c"
server_key_version: "encryptionkey@0"
I20260620 09:25:31.795419 24568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:31.810290 24568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:31.811962 24568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:31.812103 24568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:31.816031 24568 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34225
I20260620 09:25:31.816061 24635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34225 every 8 connection(s)
I20260620 09:25:31.816411 24568 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:31.817414 24636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:31.819523 24636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6: Bootstrap starting.
I20260620 09:25:31.820076 24636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:31.820348 24636 log.cc:824] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:31.820989 24636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6: No bootstrap required, opened a new log
I20260620 09:25:31.821772 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 24568
I20260620 09:25:31.821870 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:31.822083 4332 external_mini_cluster.cc:1428] Setting key 904dfa5ef1f9d35a03c4ae3b21a40d3e
I20260620 09:25:31.823144 24636 raft_consensus.cc:348] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72f36dca90e4a2fa3041647e5772ae6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34225 } }
I20260620 09:25:31.823309 24636 raft_consensus.cc:374] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:31.823348 24636 raft_consensus.cc:729] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f72f36dca90e4a2fa3041647e5772ae6, State: Initialized, Role: FOLLOWER
I20260620 09:25:31.823467 24636 consensus_queue.cc:260] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72f36dca90e4a2fa3041647e5772ae6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34225 } }
I20260620 09:25:31.823542 24636 raft_consensus.cc:388] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:31.823597 24636 raft_consensus.cc:482] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:31.823645 24636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:31.824627 24636 raft_consensus.cc:504] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72f36dca90e4a2fa3041647e5772ae6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34225 } }
I20260620 09:25:31.824736 24636 leader_election.cc:302] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f72f36dca90e4a2fa3041647e5772ae6; no voters:
I20260620 09:25:31.824872 24636 leader_election.cc:288] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:31.824928 24641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:31.825083 24636 sys_catalog.cc:564] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:31.825071 24641 raft_consensus.cc:686] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [term 1 LEADER]: Becoming Leader. State: Replica: f72f36dca90e4a2fa3041647e5772ae6, State: Running, Role: LEADER
I20260620 09:25:31.825229 24641 consensus_queue.cc:237] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72f36dca90e4a2fa3041647e5772ae6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34225 } }
I20260620 09:25:31.825529 24642 sys_catalog.cc:455] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f72f36dca90e4a2fa3041647e5772ae6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72f36dca90e4a2fa3041647e5772ae6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34225 } } }
I20260620 09:25:31.825678 24642 sys_catalog.cc:458] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:31.825873 24650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:31.826320 24650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:31.825518 24641 sys_catalog.cc:455] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f72f36dca90e4a2fa3041647e5772ae6. Latest consensus state: current_term: 1 leader_uuid: "f72f36dca90e4a2fa3041647e5772ae6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72f36dca90e4a2fa3041647e5772ae6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34225 } } }
I20260620 09:25:31.826748 24641 sys_catalog.cc:458] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:31.828179 24650 catalog_manager.cc:1347] Generated new cluster ID: 06f096a362a542ffa59ae9976f0c7aaa
I20260620 09:25:31.828245 24650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 09:25:31 dist-test-slave-414w krb5kdc[24555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 09:25:31.833683 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:31.822599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57347 (local address 127.0.0.1:34225)
0620 09:25:31.822788 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:31.822797 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:31.822829 (+ 32us) server_negotiation.cc:407] Connection header received
0620 09:25:31.823109 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:31.823116 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:31.823174 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:31.823319 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:31.824084 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:31.825521 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:31.826541 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:31.827063 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:31.827217 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:31.830014 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:31.830017 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:31.830018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:31.830028 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:31.831932 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:31.832373 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:31.832376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:31.832376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:31.832494 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:31.832743 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:31.832744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:31.832744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:31.833027 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:31.833200 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:31.833387 (+ 187us) server_negotiation.cc:299] Negotiation successful
0620 09:25:31.833483 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260620 09:25:31.834270 24650 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:31.834650 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:31.834920 24650 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:31.838043 24650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f72f36dca90e4a2fa3041647e5772ae6: Generated new TSK 0
I20260620 09:25:31.838191 24650 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:31.842406 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:31.836043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53263 (local address 127.0.0.1:34225)
0620 09:25:31.836075 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:31.836076 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:31.836184 (+ 108us) server_negotiation.cc:407] Connection header received
0620 09:25:31.836237 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:31.836238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:31.836247 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:31.836304 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:31.836903 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:31.837667 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:31.838644 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:31.838866 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:31.838941 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:31.840016 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:31.840017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:31.840017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:31.840024 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:31.841305 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:31.841642 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:31.841643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:31.841644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:31.841693 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:31.841923 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:31.841924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:31.841924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:31.842026 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:31.842143 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:31.842258 (+ 115us) server_negotiation.cc:299] Negotiation successful
0620 09:25:31.842319 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260620 09:25:31.844380 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 24568
W20260620 09:25:31.848404 24553 connection.cc:535] client connection to 127.0.0.1:34225 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34225): Connection reset by peer (error 104)
2026-06-20T09:25:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:31 dist-test-slave-414w krb5kdc[24667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:31 dist-test-slave-414w krb5kdc[24667](info): set up 2 sockets
Jun 20 09:25:31 dist-test-slave-414w krb5kdc[24667](info): commencing operation
krb5kdc: starting...
W20260620 09:25:32.881392 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:32 dist-test-slave-414w krb5kdc[24667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 09:25:32.997967 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41447
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:33.078513 24680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:33.078722 24680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:33.078760 24680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:33.079880 24680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:33.079949 24680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:33.079975 24680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:33.080031 24680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:33.080061 24680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:33.081595 24680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41447
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:33.081863 24680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:33.082094 24680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:33.086648 24688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:33.086648 24686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:33.086808 24685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:33.086898 24680 server_base.cc:1034] running on GCE node
I20260620 09:25:33.087090 24680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:33.087328 24680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:33.088487 24680 hybrid_clock.cc:648] HybridClock initialized: now 1781947533088472 us; error 27 us; skew 500 ppm
Jun 20 09:25:33 dist-test-slave-414w krb5kdc[24667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947533, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:33.095674 24680 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 09:25:33.096143 24680 webserver.cc:460] Webserver started at http://127.0.0.1:35809/ using document root <none> and password file <none>
I20260620 09:25:33.096324 24680 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:33.096362 24680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:33.096446 24680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:33.097491 24680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "b13b544dcd284ca3b72ccb833ea75f5a"
format_stamp: "Formatted at 2026-06-20 09:25:33 on dist-test-slave-414w"
server_key: "1aea39aed0fc68b7c9a3844f9f12ecbd"
server_key_iv: "fcd95a52226670d3d00440ea756f03c0"
server_key_version: "encryptionkey@0"
I20260620 09:25:33.097783 24680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "b13b544dcd284ca3b72ccb833ea75f5a"
format_stamp: "Formatted at 2026-06-20 09:25:33 on dist-test-slave-414w"
server_key: "1aea39aed0fc68b7c9a3844f9f12ecbd"
server_key_iv: "fcd95a52226670d3d00440ea756f03c0"
server_key_version: "encryptionkey@0"
I20260620 09:25:33.099025 24680 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:25:33.099699 24695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:33.099895 24680 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:33.099977 24680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "b13b544dcd284ca3b72ccb833ea75f5a"
format_stamp: "Formatted at 2026-06-20 09:25:33 on dist-test-slave-414w"
server_key: "1aea39aed0fc68b7c9a3844f9f12ecbd"
server_key_iv: "fcd95a52226670d3d00440ea756f03c0"
server_key_version: "encryptionkey@0"
I20260620 09:25:33.100031 24680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:33.112171 24680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:33.114130 24680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:33.114233 24680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:33.117998 24680 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41447
I20260620 09:25:33.118052 24747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41447 every 8 connection(s)
I20260620 09:25:33.118364 24680 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:33.119294 24748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:33.121510 24748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a: Bootstrap starting.
I20260620 09:25:33.122061 24748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:33.122354 24748 log.cc:824] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:33.122383 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 24680
I20260620 09:25:33.122543 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:33.122740 4332 external_mini_cluster.cc:1428] Setting key 30c01384fad6429de389ae65b538c697
I20260620 09:25:33.123075 24748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a: No bootstrap required, opened a new log
I20260620 09:25:33.125793 24748 raft_consensus.cc:348] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13b544dcd284ca3b72ccb833ea75f5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } }
I20260620 09:25:33.125950 24748 raft_consensus.cc:374] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:33.125980 24748 raft_consensus.cc:729] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13b544dcd284ca3b72ccb833ea75f5a, State: Initialized, Role: FOLLOWER
I20260620 09:25:33.126111 24748 consensus_queue.cc:260] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13b544dcd284ca3b72ccb833ea75f5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } }
I20260620 09:25:33.126185 24748 raft_consensus.cc:388] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:33.126222 24748 raft_consensus.cc:482] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:33.126314 24748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:33.126866 24748 raft_consensus.cc:504] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13b544dcd284ca3b72ccb833ea75f5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } }
I20260620 09:25:33.126981 24748 leader_election.cc:302] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b13b544dcd284ca3b72ccb833ea75f5a; no voters:
I20260620 09:25:33.127163 24748 leader_election.cc:288] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:33.127187 24753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:33.127341 24753 raft_consensus.cc:686] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [term 1 LEADER]: Becoming Leader. State: Replica: b13b544dcd284ca3b72ccb833ea75f5a, State: Running, Role: LEADER
I20260620 09:25:33.127432 24748 sys_catalog.cc:564] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:33.127498 24753 consensus_queue.cc:237] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13b544dcd284ca3b72ccb833ea75f5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } }
I20260620 09:25:33.127930 24754 sys_catalog.cc:455] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b13b544dcd284ca3b72ccb833ea75f5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13b544dcd284ca3b72ccb833ea75f5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } } }
I20260620 09:25:33.127933 24755 sys_catalog.cc:455] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b13b544dcd284ca3b72ccb833ea75f5a. Latest consensus state: current_term: 1 leader_uuid: "b13b544dcd284ca3b72ccb833ea75f5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13b544dcd284ca3b72ccb833ea75f5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } } }
I20260620 09:25:33.128088 24754 sys_catalog.cc:458] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:33.128100 24755 sys_catalog.cc:458] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:33.128347 24762 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:33.128749 24752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:33.123214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44931 (local address 127.0.0.1:41447)
0620 09:25:33.123392 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:33.123400 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:33.123425 (+ 25us) server_negotiation.cc:407] Connection header received
0620 09:25:33.123856 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:33.123862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:33.123904 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:33.124026 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:33.124269 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:33.125111 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:33.126256 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:33.126728 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:33.126815 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:33.127552 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:33.127555 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:33.127555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:33.127756 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:33.127792 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:33.127805 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:33.128322 (+ 517us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:33.128537 (+ 215us) server_negotiation.cc:299] Negotiation successful
0620 09:25:33.128566 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260620 09:25:33.129094 24762 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:33.129973 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:33.130837 24762 catalog_manager.cc:1347] Generated new cluster ID: 0a1af9943532465cbbcae9d2b2115dd2
I20260620 09:25:33.130884 24762 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:33.134779 24752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:33.131372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53367 (local address 127.0.0.1:41447)
0620 09:25:33.131405 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:33.131407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:33.131518 (+ 111us) server_negotiation.cc:407] Connection header received
0620 09:25:33.131572 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:33.131572 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:33.131582 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:33.131636 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:33.131776 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:33.132467 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:33.133530 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:33.133835 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:33.133893 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:33.134529 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:33.134530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:33.134530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:33.134534 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:33.134547 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:33.134550 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:33.134605 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:33.134705 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 09:25:33.134714 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 09:25:33.151489 24762 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:33.152184 24762 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:33.155179 24762 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b13b544dcd284ca3b72ccb833ea75f5a: Generated new TSK 0
I20260620 09:25:33.155332 24762 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:33.158818 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 24680
W20260620 09:25:33.163156 24665 connection.cc:535] client connection to 127.0.0.1:41447 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41447): Connection reset by peer (error 104)
2026-06-20T09:25:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:33 dist-test-slave-414w krb5kdc[24779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:33 dist-test-slave-414w krb5kdc[24779](info): set up 2 sockets
Jun 20 09:25:33 dist-test-slave-414w krb5kdc[24779](info): commencing operation
krb5kdc: starting...
W20260620 09:25:34.193486 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:34 dist-test-slave-414w krb5kdc[24779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 09:25:34.307840 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39263
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:34.385838 24792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:34.386025 24792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:34.386047 24792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:34.387194 24792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:34.387236 24792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:34.387250 24792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:34.387264 24792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:34.387275 24792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:34.388723 24792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39263
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:34.388960 24792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:34.389210 24792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:34.394284 24797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:34.394304 24792 server_base.cc:1034] running on GCE node
W20260620 09:25:34.394289 24800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:34.394299 24798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:34.394719 24792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:34.395001 24792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:34.396162 24792 hybrid_clock.cc:648] HybridClock initialized: now 1781947534396142 us; error 52 us; skew 500 ppm
Jun 20 09:25:34 dist-test-slave-414w krb5kdc[24779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947534, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:34.398577 24792 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 09:25:34.399044 24792 webserver.cc:460] Webserver started at http://127.0.0.1:37121/ using document root <none> and password file <none>
I20260620 09:25:34.399227 24792 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:34.399267 24792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:34.399353 24792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:34.400295 24792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "eb6d2d87bd064728b3ee100a2704d5a4"
format_stamp: "Formatted at 2026-06-20 09:25:34 on dist-test-slave-414w"
server_key: "a26c35d18dc265be76c02207e1c68694"
server_key_iv: "168b43b29516e7e1dbca049c450286b7"
server_key_version: "encryptionkey@0"
I20260620 09:25:34.400573 24792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "eb6d2d87bd064728b3ee100a2704d5a4"
format_stamp: "Formatted at 2026-06-20 09:25:34 on dist-test-slave-414w"
server_key: "a26c35d18dc265be76c02207e1c68694"
server_key_iv: "168b43b29516e7e1dbca049c450286b7"
server_key_version: "encryptionkey@0"
I20260620 09:25:34.401835 24792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:25:34.402464 24807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:34.402643 24792 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:34.402709 24792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "eb6d2d87bd064728b3ee100a2704d5a4"
format_stamp: "Formatted at 2026-06-20 09:25:34 on dist-test-slave-414w"
server_key: "a26c35d18dc265be76c02207e1c68694"
server_key_iv: "168b43b29516e7e1dbca049c450286b7"
server_key_version: "encryptionkey@0"
I20260620 09:25:34.402755 24792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:34.405871 24792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:34.407966 24792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:34.408064 24792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:34.411923 24792 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39263
I20260620 09:25:34.411952 24859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39263 every 8 connection(s)
I20260620 09:25:34.412273 24792 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:34.413203 24860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:34.415508 24860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4: Bootstrap starting.
I20260620 09:25:34.416003 24860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:34.416227 24860 log.cc:824] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:34.416812 24860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4: No bootstrap required, opened a new log
I20260620 09:25:34.418390 24860 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb6d2d87bd064728b3ee100a2704d5a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39263 } }
I20260620 09:25:34.418541 24860 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:34.418565 24860 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb6d2d87bd064728b3ee100a2704d5a4, State: Initialized, Role: FOLLOWER
I20260620 09:25:34.418658 24860 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb6d2d87bd064728b3ee100a2704d5a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39263 } }
I20260620 09:25:34.418706 24860 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:34.418727 24860 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:34.418756 24860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:34.419210 24860 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb6d2d87bd064728b3ee100a2704d5a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39263 } }
I20260620 09:25:34.419291 24860 leader_election.cc:302] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb6d2d87bd064728b3ee100a2704d5a4; no voters:
I20260620 09:25:34.419445 24860 leader_election.cc:288] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:34.419463 24863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:34.419576 24863 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [term 1 LEADER]: Becoming Leader. State: Replica: eb6d2d87bd064728b3ee100a2704d5a4, State: Running, Role: LEADER
I20260620 09:25:34.419660 24863 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb6d2d87bd064728b3ee100a2704d5a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39263 } }
I20260620 09:25:34.419697 24860 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:34.420101 24864 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb6d2d87bd064728b3ee100a2704d5a4. Latest consensus state: current_term: 1 leader_uuid: "eb6d2d87bd064728b3ee100a2704d5a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb6d2d87bd064728b3ee100a2704d5a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39263 } } }
I20260620 09:25:34.420301 24864 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:34.420243 24865 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb6d2d87bd064728b3ee100a2704d5a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb6d2d87bd064728b3ee100a2704d5a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39263 } } }
I20260620 09:25:34.420557 24865 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:34.420769 24872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:34.421293 24872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:34.421749 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 24792
I20260620 09:25:34.421868 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:34.422020 4332 external_mini_cluster.cc:1428] Setting key 88461ffba7e84f945cea082dcbecacbe
I20260620 09:25:34.423237 24872 catalog_manager.cc:1347] Generated new cluster ID: 4cbed2328a8844a8b09f0735b9ef0085
I20260620 09:25:34.423291 24872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:34.426401 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:34.422591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50117 (local address 127.0.0.1:39263)
0620 09:25:34.423027 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:34.423035 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:34.423064 (+ 29us) server_negotiation.cc:407] Connection header received
0620 09:25:34.423171 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:34.423178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:34.425224 (+ 2046us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:34.425384 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:34.425986 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:34.425989 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:34.425989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:34.425993 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:34.426036 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:34.426047 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:34.426118 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:34.426188 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 09:25:34.426228 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":70,"threads_started":1}
I20260620 09:25:34.427273 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:34.434288 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:34.429444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50531 (local address 127.0.0.1:39263)
0620 09:25:34.429479 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:34.429481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:34.433314 (+ 3833us) server_negotiation.cc:407] Connection header received
0620 09:25:34.433384 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:34.433385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:34.433390 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:34.433455 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:34.434058 (+ 603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:34.434059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:34.434060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:34.434064 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:34.434077 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:34.434081 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:34.434137 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:34.434192 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 09:25:34.434206 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:25:34.435552 24872 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:34.435696 24872 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:34.448071 24872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb6d2d87bd064728b3ee100a2704d5a4: Generated new TSK 0
I20260620 09:25:34.448231 24872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:34.453388 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 24792
2026-06-20T09:25:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1294 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:34 dist-test-slave-414w krb5kdc[24891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:34 dist-test-slave-414w krb5kdc[24891](info): set up 2 sockets
Jun 20 09:25:34 dist-test-slave-414w krb5kdc[24891](info): commencing operation
krb5kdc: starting...
W20260620 09:25:35.481410 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:35 dist-test-slave-414w krb5kdc[24891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 09:25:35.592962 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34883
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:35.675844 24904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:35.676083 24904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:35.676123 24904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:35.677269 24904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:35.677337 24904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:35.677363 24904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:35.677403 24904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:35.677426 24904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:35.678876 24904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:35.679149 24904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:35.679379 24904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:35.684231 24910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:35.684217 24912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:35.684217 24909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:35.684245 24904 server_base.cc:1034] running on GCE node
I20260620 09:25:35.684767 24904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:35.685025 24904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:35.686170 24904 hybrid_clock.cc:648] HybridClock initialized: now 1781947535686173 us; error 60 us; skew 500 ppm
Jun 20 09:25:35 dist-test-slave-414w krb5kdc[24891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947535, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:35.688619 24904 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 09:25:35.689090 24904 webserver.cc:460] Webserver started at http://127.0.0.1:34767/ using document root <none> and password file <none>
I20260620 09:25:35.689335 24904 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:35.689404 24904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:35.689556 24904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:35.690589 24904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "4e33a57569794e16ab3f54b649e929cf"
format_stamp: "Formatted at 2026-06-20 09:25:35 on dist-test-slave-414w"
server_key: "97296666d7d680c5f2d735f6f64dbcf1"
server_key_iv: "cb33e5374289971a029de31d4b13ad7c"
server_key_version: "encryptionkey@0"
I20260620 09:25:35.690941 24904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "4e33a57569794e16ab3f54b649e929cf"
format_stamp: "Formatted at 2026-06-20 09:25:35 on dist-test-slave-414w"
server_key: "97296666d7d680c5f2d735f6f64dbcf1"
server_key_iv: "cb33e5374289971a029de31d4b13ad7c"
server_key_version: "encryptionkey@0"
I20260620 09:25:35.692210 24904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:25:35.692921 24919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:35.693223 24904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:35.693320 24904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "4e33a57569794e16ab3f54b649e929cf"
format_stamp: "Formatted at 2026-06-20 09:25:35 on dist-test-slave-414w"
server_key: "97296666d7d680c5f2d735f6f64dbcf1"
server_key_iv: "cb33e5374289971a029de31d4b13ad7c"
server_key_version: "encryptionkey@0"
I20260620 09:25:35.693418 24904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:35.702582 24904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:35.704413 24904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:35.704586 24904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:35.709010 24904 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34883
I20260620 09:25:35.709043 24971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34883 every 8 connection(s)
I20260620 09:25:35.709438 24904 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:35.710345 24972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:35.712481 24972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf: Bootstrap starting.
I20260620 09:25:35.713052 24972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:35.713351 24972 log.cc:824] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:35.714011 24972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf: No bootstrap required, opened a new log
I20260620 09:25:35.715518 24972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e33a57569794e16ab3f54b649e929cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } }
I20260620 09:25:35.715658 24972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:35.715679 24972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e33a57569794e16ab3f54b649e929cf, State: Initialized, Role: FOLLOWER
I20260620 09:25:35.715833 24972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e33a57569794e16ab3f54b649e929cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } }
I20260620 09:25:35.715891 24972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:35.715911 24972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:35.715971 24972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:35.716470 24972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e33a57569794e16ab3f54b649e929cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } }
I20260620 09:25:35.716586 24972 leader_election.cc:302] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e33a57569794e16ab3f54b649e929cf; no voters:
I20260620 09:25:35.716755 24972 leader_election.cc:288] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:35.716791 24975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:35.716966 24975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [term 1 LEADER]: Becoming Leader. State: Replica: 4e33a57569794e16ab3f54b649e929cf, State: Running, Role: LEADER
I20260620 09:25:35.716996 24972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:35.717053 24975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e33a57569794e16ab3f54b649e929cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } }
I20260620 09:25:35.717443 24976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e33a57569794e16ab3f54b649e929cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e33a57569794e16ab3f54b649e929cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } } }
I20260620 09:25:35.717445 24977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e33a57569794e16ab3f54b649e929cf. Latest consensus state: current_term: 1 leader_uuid: "4e33a57569794e16ab3f54b649e929cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e33a57569794e16ab3f54b649e929cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } } }
I20260620 09:25:35.717576 24976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:35.717638 24977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:35.717706 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 24904
I20260620 09:25:35.717829 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:35.717981 24979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:35.718009 4332 external_mini_cluster.cc:1428] Setting key bd034c4cfdfcaaefd8fd1fdcdc6796db
I20260620 09:25:35.718484 24979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:35.720094 24979 catalog_manager.cc:1347] Generated new cluster ID: 5c79252df68446b68c3fdafe2530d12e
I20260620 09:25:35.720148 24979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 09:25:35 dist-test-slave-414w krb5kdc[24891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 09:25:35.730217 24988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:35.718637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44705 (local address 127.0.0.1:34883)
0620 09:25:35.718803 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:35.718812 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:35.718984 (+ 172us) server_negotiation.cc:407] Connection header received
0620 09:25:35.719210 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:35.719218 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:35.719270 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:35.719459 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:35.720350 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:35.721755 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:35.722772 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:35.723408 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:35.723523 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:35.726429 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:35.726432 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:35.726433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:35.726445 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:35.728491 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:35.728886 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:35.728889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:35.728889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:35.728991 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:35.729231 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:35.729232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:35.729232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:35.729507 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:35.729656 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:35.729778 (+ 122us) server_negotiation.cc:299] Negotiation successful
0620 09:25:35.729995 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260620 09:25:35.731069 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:35.732023 24979 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:35.732710 24979 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:35.739620 24988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:35.732443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50773 (local address 127.0.0.1:34883)
0620 09:25:35.732483 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:35.732485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:35.732634 (+ 149us) server_negotiation.cc:407] Connection header received
0620 09:25:35.732691 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:35.732692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:35.732702 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:35.732769 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:35.733415 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:35.734136 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:35.735161 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:35.735470 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:35.735527 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:35.736820 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:35.736821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:35.736821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:35.736828 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:35.738373 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:35.738859 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:35.738860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:35.738860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:35.738911 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:35.739149 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:35.739150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:35.739150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:35.739253 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:35.739341 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:35.739480 (+ 139us) server_negotiation.cc:299] Negotiation successful
0620 09:25:35.739543 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 09:25:35.743130 24979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e33a57569794e16ab3f54b649e929cf: Generated new TSK 0
I20260620 09:25:35.743295 24979 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:35.747033 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 24904
2026-06-20T09:25:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1292 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:35 dist-test-slave-414w krb5kdc[25003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:35 dist-test-slave-414w krb5kdc[25003](info): set up 2 sockets
Jun 20 09:25:35 dist-test-slave-414w krb5kdc[25003](info): commencing operation
krb5kdc: starting...
W20260620 09:25:36.773504 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:36 dist-test-slave-414w krb5kdc[25003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 09:25:36.890208 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33433
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:36.979205 25016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:36.979519 25016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:36.979597 25016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:36.982075 25016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:36.982154 25016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:36.982184 25016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:36.982208 25016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:36.982283 25016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:36.984758 25016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:36.985050 25016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:36.985339 25016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:36.990764 25022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:36.990784 25024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:36.990800 25021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:36.991285 25016 server_base.cc:1034] running on GCE node
I20260620 09:25:36.991549 25016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:36.991851 25016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:36.993019 25016 hybrid_clock.cc:648] HybridClock initialized: now 1781947536992993 us; error 43 us; skew 500 ppm
Jun 20 09:25:36 dist-test-slave-414w krb5kdc[25003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947536, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:36.996268 25016 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 09:25:36.996762 25016 webserver.cc:460] Webserver started at http://127.0.0.1:42655/ using document root <none> and password file <none>
I20260620 09:25:36.996994 25016 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:36.997061 25016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:36.997226 25016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:36.998481 25016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "065b4933bb3b46b69249b6682fe0dd71"
format_stamp: "Formatted at 2026-06-20 09:25:36 on dist-test-slave-414w"
server_key: "01355d492d26a277b6d0c3f340921265"
server_key_iv: "b8f60307a8cf03da90de125b8cde118b"
server_key_version: "encryptionkey@0"
I20260620 09:25:36.998845 25016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "065b4933bb3b46b69249b6682fe0dd71"
format_stamp: "Formatted at 2026-06-20 09:25:36 on dist-test-slave-414w"
server_key: "01355d492d26a277b6d0c3f340921265"
server_key_iv: "b8f60307a8cf03da90de125b8cde118b"
server_key_version: "encryptionkey@0"
I20260620 09:25:37.000442 25016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:25:37.001286 25031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:37.001475 25016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:25:37.001569 25016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "065b4933bb3b46b69249b6682fe0dd71"
format_stamp: "Formatted at 2026-06-20 09:25:36 on dist-test-slave-414w"
server_key: "01355d492d26a277b6d0c3f340921265"
server_key_iv: "b8f60307a8cf03da90de125b8cde118b"
server_key_version: "encryptionkey@0"
I20260620 09:25:37.001657 25016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:37.015328 25016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:37.017132 25016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:37.017336 25016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:37.020949 25016 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33433
I20260620 09:25:37.020983 25083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33433 every 8 connection(s)
I20260620 09:25:37.021378 25016 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:37.022302 25084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:37.024384 25084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71: Bootstrap starting.
I20260620 09:25:37.024876 25084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:37.025110 25084 log.cc:824] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:37.025461 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 25016
I20260620 09:25:37.025578 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:37.025739 4332 external_mini_cluster.cc:1428] Setting key 2b1f7763070c885d9cfae9d96ab8384f
I20260620 09:25:37.025940 25084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71: No bootstrap required, opened a new log
I20260620 09:25:37.027659 25084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "065b4933bb3b46b69249b6682fe0dd71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33433 } }
I20260620 09:25:37.027813 25084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:37.027876 25084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 065b4933bb3b46b69249b6682fe0dd71, State: Initialized, Role: FOLLOWER
I20260620 09:25:37.028010 25084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "065b4933bb3b46b69249b6682fe0dd71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33433 } }
I20260620 09:25:37.028090 25084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:37.028131 25084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:37.028177 25084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:37.028689 25084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "065b4933bb3b46b69249b6682fe0dd71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33433 } }
I20260620 09:25:37.028805 25084 leader_election.cc:302] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 065b4933bb3b46b69249b6682fe0dd71; no voters:
I20260620 09:25:37.029000 25084 leader_election.cc:288] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:37.029028 25089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:37.029134 25089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [term 1 LEADER]: Becoming Leader. State: Replica: 065b4933bb3b46b69249b6682fe0dd71, State: Running, Role: LEADER
I20260620 09:25:37.029274 25084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:37.029294 25089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "065b4933bb3b46b69249b6682fe0dd71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33433 } }
I20260620 09:25:37.029721 25090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "065b4933bb3b46b69249b6682fe0dd71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "065b4933bb3b46b69249b6682fe0dd71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33433 } } }
I20260620 09:25:37.029971 25090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:37.029894 25091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 065b4933bb3b46b69249b6682fe0dd71. Latest consensus state: current_term: 1 leader_uuid: "065b4933bb3b46b69249b6682fe0dd71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "065b4933bb3b46b69249b6682fe0dd71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33433 } } }
I20260620 09:25:37.030211 25091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:37.030381 25098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:37.030866 25098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:37.032440 25098 catalog_manager.cc:1347] Generated new cluster ID: 2bafe614963c4c4792178dc0705c8244
I20260620 09:25:37.032490 25098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:37.032830 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:37.026241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44571 (local address 127.0.0.1:33433)
0620 09:25:37.026423 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:37.026431 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:37.026464 (+ 33us) server_negotiation.cc:407] Connection header received
0620 09:25:37.027652 (+ 1188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:37.027659 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:37.027707 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:37.027868 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:37.028064 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:37.029468 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:37.030609 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:37.031205 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:37.031318 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:37.032011 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:37.032015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:37.032016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:37.032250 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:37.032285 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:37.032299 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:37.032357 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:37.032634 (+ 277us) server_negotiation.cc:299] Negotiation successful
0620 09:25:37.032682 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260620 09:25:37.036949 25098 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:37.037587 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:37.037714 25098 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:37.048300 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:37.045496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55469 (local address 127.0.0.1:33433)
0620 09:25:37.045533 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:37.045535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:37.045678 (+ 143us) server_negotiation.cc:407] Connection header received
0620 09:25:37.045752 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:37.045753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:37.045763 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:37.045813 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:37.045932 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:37.046388 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:37.047134 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:37.047326 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:37.047412 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:37.048057 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:37.048058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:37.048058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:37.048063 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:37.048075 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:37.048079 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:37.048133 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:37.048193 (+ 60us) server_negotiation.cc:299] Negotiation successful
0620 09:25:37.048214 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:25:37.050194 25098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 065b4933bb3b46b69249b6682fe0dd71: Generated new TSK 0
I20260620 09:25:37.050383 25098 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:37.055629 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 25016
2026-06-20T09:25:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1308 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:37 dist-test-slave-414w krb5kdc[25115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:37 dist-test-slave-414w krb5kdc[25115](info): set up 2 sockets
Jun 20 09:25:37 dist-test-slave-414w krb5kdc[25115](info): commencing operation
krb5kdc: starting...
W20260620 09:25:38.093475 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:38 dist-test-slave-414w krb5kdc[25115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 09:25:38.204012 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42753
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:38.282245 25128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:38.282481 25128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:38.282517 25128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:38.283638 25128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:38.283702 25128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:38.283727 25128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:38.283754 25128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:38.283777 25128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:38.285215 25128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:38.285481 25128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:38.285694 25128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:38.290275 25133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:38.290382 25128 server_base.cc:1034] running on GCE node
W20260620 09:25:38.290270 25136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:38.290335 25134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:38.290788 25128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:38.291054 25128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:38.292209 25128 hybrid_clock.cc:648] HybridClock initialized: now 1781947538292208 us; error 35 us; skew 500 ppm
Jun 20 09:25:38 dist-test-slave-414w krb5kdc[25115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947538, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:38.300011 25128 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 09:25:38.300500 25128 webserver.cc:460] Webserver started at http://127.0.0.1:44631/ using document root <none> and password file <none>
I20260620 09:25:38.300745 25128 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:38.300815 25128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:38.300961 25128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:38.302033 25128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "9d43277151e9458fabd36ff14cf6347c"
format_stamp: "Formatted at 2026-06-20 09:25:38 on dist-test-slave-414w"
server_key: "ac6c02347f663bc3672c9ce2cc9d500c"
server_key_iv: "cb46374975de9abccb0fd20771c8b2bf"
server_key_version: "encryptionkey@0"
I20260620 09:25:38.302397 25128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "9d43277151e9458fabd36ff14cf6347c"
format_stamp: "Formatted at 2026-06-20 09:25:38 on dist-test-slave-414w"
server_key: "ac6c02347f663bc3672c9ce2cc9d500c"
server_key_iv: "cb46374975de9abccb0fd20771c8b2bf"
server_key_version: "encryptionkey@0"
I20260620 09:25:38.303694 25128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:25:38.304450 25143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:38.304656 25128 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:38.304760 25128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "9d43277151e9458fabd36ff14cf6347c"
format_stamp: "Formatted at 2026-06-20 09:25:38 on dist-test-slave-414w"
server_key: "ac6c02347f663bc3672c9ce2cc9d500c"
server_key_iv: "cb46374975de9abccb0fd20771c8b2bf"
server_key_version: "encryptionkey@0"
I20260620 09:25:38.304845 25128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:38.308130 25128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:38.310389 25128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:38.310547 25128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:38.314607 25128 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42753
I20260620 09:25:38.314637 25195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42753 every 8 connection(s)
I20260620 09:25:38.314954 25128 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:38.315779 25196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:38.317865 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 25128
I20260620 09:25:38.317991 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:38.318161 4332 external_mini_cluster.cc:1428] Setting key 8646281e554c11e94d06b6c8e6b77a26
I20260620 09:25:38.318217 25196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c: Bootstrap starting.
I20260620 09:25:38.319039 25196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:38.319351 25196 log.cc:824] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:38.320134 25196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c: No bootstrap required, opened a new log
I20260620 09:25:38.320859 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:38.318735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52703 (local address 127.0.0.1:42753)
0620 09:25:38.318930 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:38.318939 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:38.318974 (+ 35us) server_negotiation.cc:407] Connection header received
0620 09:25:38.319611 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:38.319618 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:38.319642 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:38.319753 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:38.320366 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:38.320369 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:38.320369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:38.320373 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:38.320423 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:38.320436 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:38.320522 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:38.320630 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 09:25:38.320676 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":80,"threads_started":1}
I20260620 09:25:38.321934 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:38.322705 25196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d43277151e9458fabd36ff14cf6347c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42753 } }
I20260620 09:25:38.322886 25196 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:38.322921 25196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d43277151e9458fabd36ff14cf6347c, State: Initialized, Role: FOLLOWER
I20260620 09:25:38.323041 25196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d43277151e9458fabd36ff14cf6347c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42753 } }
I20260620 09:25:38.323148 25196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:38.323196 25196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:38.323251 25196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:38.323868 25196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d43277151e9458fabd36ff14cf6347c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42753 } }
I20260620 09:25:38.323994 25196 leader_election.cc:302] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d43277151e9458fabd36ff14cf6347c; no voters:
I20260620 09:25:38.324137 25196 leader_election.cc:288] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:38.324178 25206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:38.324316 25206 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [term 1 LEADER]: Becoming Leader. State: Replica: 9d43277151e9458fabd36ff14cf6347c, State: Running, Role: LEADER
I20260620 09:25:38.324368 25196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:38.324445 25206 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d43277151e9458fabd36ff14cf6347c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42753 } }
I20260620 09:25:38.324577 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:38.323529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45363 (local address 127.0.0.1:42753)
0620 09:25:38.323568 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:38.323570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:38.323632 (+ 62us) server_negotiation.cc:407] Connection header received
0620 09:25:38.323702 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:38.323703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:38.323707 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:38.323761 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:38.324305 (+ 544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:38.324306 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:38.324306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:38.324310 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:38.324320 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:38.324324 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:38.324366 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:38.324508 (+ 142us) server_negotiation.cc:299] Negotiation successful
0620 09:25:38.324520 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 09:25:38.324800 25206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d43277151e9458fabd36ff14cf6347c. Latest consensus state: current_term: 1 leader_uuid: "9d43277151e9458fabd36ff14cf6347c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d43277151e9458fabd36ff14cf6347c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42753 } } }
I20260620 09:25:38.324905 25206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:38.325173 25214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:38.324810 25208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d43277151e9458fabd36ff14cf6347c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d43277151e9458fabd36ff14cf6347c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42753 } } }
I20260620 09:25:38.325416 25208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:38.325825 25214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:38.327418 25214 catalog_manager.cc:1347] Generated new cluster ID: 16ff3d1e96c049d8be2a5b29ee47d8ae
I20260620 09:25:38.327489 25214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:38.336222 25214 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:38.336364 25214 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:38.347191 25214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d43277151e9458fabd36ff14cf6347c: Generated new TSK 0
I20260620 09:25:38.347465 25214 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:38.353477 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 25128
2026-06-20T09:25:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:38 dist-test-slave-414w krb5kdc[25227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:38 dist-test-slave-414w krb5kdc[25227](info): set up 2 sockets
Jun 20 09:25:38 dist-test-slave-414w krb5kdc[25227](info): commencing operation
krb5kdc: starting...
W20260620 09:25:39.389444 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:39 dist-test-slave-414w krb5kdc[25227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 09:25:39.500003 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43019
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:39.578351 25240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:39.578536 25240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:39.578557 25240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:39.579813 25240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:39.579854 25240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:39.579867 25240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:39.579900 25240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:39.579926 25240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:39.581418 25240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:39.581652 25240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:39.581877 25240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:39.586385 25245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:39.586441 25246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:39.586516 25240 server_base.cc:1034] running on GCE node
W20260620 09:25:39.586397 25248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:39.586858 25240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:39.587126 25240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:39.588313 25240 hybrid_clock.cc:648] HybridClock initialized: now 1781947539588292 us; error 37 us; skew 500 ppm
Jun 20 09:25:39 dist-test-slave-414w krb5kdc[25227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947539, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:39.590713 25240 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 09:25:39.591118 25240 webserver.cc:460] Webserver started at http://127.0.0.1:34705/ using document root <none> and password file <none>
I20260620 09:25:39.591348 25240 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:39.591415 25240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:39.591560 25240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:39.592559 25240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "cf2970a616d34f2da72df1f5bab387be"
format_stamp: "Formatted at 2026-06-20 09:25:39 on dist-test-slave-414w"
server_key: "a10ee920fb4b74e5d495a8a48e815db7"
server_key_iv: "b1f16d89fa4d99db3bf8f7482c3410a3"
server_key_version: "encryptionkey@0"
I20260620 09:25:39.592898 25240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "cf2970a616d34f2da72df1f5bab387be"
format_stamp: "Formatted at 2026-06-20 09:25:39 on dist-test-slave-414w"
server_key: "a10ee920fb4b74e5d495a8a48e815db7"
server_key_iv: "b1f16d89fa4d99db3bf8f7482c3410a3"
server_key_version: "encryptionkey@0"
I20260620 09:25:39.594298 25240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 09:25:39.595031 25255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:39.595224 25240 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:39.595322 25240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "cf2970a616d34f2da72df1f5bab387be"
format_stamp: "Formatted at 2026-06-20 09:25:39 on dist-test-slave-414w"
server_key: "a10ee920fb4b74e5d495a8a48e815db7"
server_key_iv: "b1f16d89fa4d99db3bf8f7482c3410a3"
server_key_version: "encryptionkey@0"
I20260620 09:25:39.595407 25240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:39.625988 25240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:39.627873 25240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:39.628057 25240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:39.631880 25240 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43019
I20260620 09:25:39.631917 25307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43019 every 8 connection(s)
I20260620 09:25:39.632290 25240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:39.633411 25308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:39.634290 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 25240
I20260620 09:25:39.634405 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:39.634595 4332 external_mini_cluster.cc:1428] Setting key 8b24c30ad1615ecffebf828ea4ab779d
I20260620 09:25:39.636698 25308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be: Bootstrap starting.
I20260620 09:25:39.637333 25308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:39.637614 25308 log.cc:824] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:39.638358 25308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be: No bootstrap required, opened a new log
I20260620 09:25:39.640767 25308 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2970a616d34f2da72df1f5bab387be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43019 } }
I20260620 09:25:39.640928 25308 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:39.640952 25308 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf2970a616d34f2da72df1f5bab387be, State: Initialized, Role: FOLLOWER
I20260620 09:25:39.641079 25308 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2970a616d34f2da72df1f5bab387be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43019 } }
I20260620 09:25:39.641192 25308 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:39.641234 25308 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:39.641287 25308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [term 0 FOLLOWER]: Advancing to term 1
Jun 20 09:25:39 dist-test-slave-414w krb5kdc[25227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 09:25:39.641848 25308 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2970a616d34f2da72df1f5bab387be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43019 } }
I20260620 09:25:39.641964 25308 leader_election.cc:302] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf2970a616d34f2da72df1f5bab387be; no voters:
I20260620 09:25:39.642182 25308 leader_election.cc:288] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:39.642213 25313 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:39.642328 25313 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [term 1 LEADER]: Becoming Leader. State: Replica: cf2970a616d34f2da72df1f5bab387be, State: Running, Role: LEADER
I20260620 09:25:39.642436 25308 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:39.642454 25313 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2970a616d34f2da72df1f5bab387be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43019 } }
I20260620 09:25:39.642850 25314 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf2970a616d34f2da72df1f5bab387be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2970a616d34f2da72df1f5bab387be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43019 } } }
I20260620 09:25:39.642942 25314 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:39.643093 25315 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf2970a616d34f2da72df1f5bab387be. Latest consensus state: current_term: 1 leader_uuid: "cf2970a616d34f2da72df1f5bab387be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2970a616d34f2da72df1f5bab387be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43019 } } }
I20260620 09:25:39.643154 25315 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:39.643532 25320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:39.644088 25320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:39.645946 25320 catalog_manager.cc:1347] Generated new cluster ID: 4f631ae7246047fd9f4f8f72eb2795a2
I20260620 09:25:39.646035 25320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:39.646121 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:39.635120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.173:59117 (local address 127.0.0.1:43019)
0620 09:25:39.635274 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:39.635282 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:39.635315 (+ 33us) server_negotiation.cc:407] Connection header received
0620 09:25:39.635715 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:39.635721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:39.635766 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:39.635873 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:39.636670 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:39.637748 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:39.638712 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:39.639110 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:39.639208 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:39.642019 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:39.642023 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:39.642023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:39.642042 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:39.644169 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:39.644742 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:39.644744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:39.644744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:39.644838 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:39.645105 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:39.645106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:39.645106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:39.645442 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:39.645598 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:39.645758 (+ 160us) server_negotiation.cc:299] Negotiation successful
0620 09:25:39.645867 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260620 09:25:39.647006 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:39.654274 25320 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:39.654915 25320 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:39.655992 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:39.648445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.173:37581 (local address 127.0.0.1:43019)
0620 09:25:39.648484 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:39.648486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:39.648581 (+ 95us) server_negotiation.cc:407] Connection header received
0620 09:25:39.648619 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:39.648620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:39.648629 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:39.648672 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:39.649281 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:39.650259 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:39.651254 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:39.651571 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:39.651629 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:39.652964 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:39.652965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:39.652965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:39.652972 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:39.654666 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:39.655091 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:39.655092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:39.655092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:39.655145 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:39.655380 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:39.655380 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:39.655381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:39.655489 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:39.655596 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:39.655755 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 09:25:39.655821 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 09:25:39.659417 25320 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf2970a616d34f2da72df1f5bab387be: Generated new TSK 0
I20260620 09:25:39.659586 25320 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:39.664706 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 25240
W20260620 09:25:39.668645 25225 connection.cc:535] client connection to 127.0.0.1:43019 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43019): Connection reset by peer (error 104)
2026-06-20T09:25:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1311 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:39 dist-test-slave-414w krb5kdc[25339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:39 dist-test-slave-414w krb5kdc[25339](info): set up 2 sockets
Jun 20 09:25:39 dist-test-slave-414w krb5kdc[25339](info): commencing operation
krb5kdc: starting...
W20260620 09:25:40.697432 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:40 dist-test-slave-414w krb5kdc[25339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 09:25:40.808123 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41829
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:40.886317 25352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:40.886536 25352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:40.886572 25352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:40.887689 25352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:40.887754 25352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:40.887787 25352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:40.887818 25352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:40.887843 25352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:40.889313 25352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:40.889583 25352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:40.889797 25352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:40.894342 25360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:40.894344 25357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:40.894411 25358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:40.894496 25352 server_base.cc:1034] running on GCE node
I20260620 09:25:40.894820 25352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:40.895087 25352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:40.896245 25352 hybrid_clock.cc:648] HybridClock initialized: now 1781947540896239 us; error 34 us; skew 500 ppm
Jun 20 09:25:40 dist-test-slave-414w krb5kdc[25339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947540, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:40.898554 25352 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 09:25:40.898978 25352 webserver.cc:460] Webserver started at http://127.0.0.1:45707/ using document root <none> and password file <none>
I20260620 09:25:40.899202 25352 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:40.899266 25352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:40.899406 25352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:40.900357 25352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "e20954508c0a41ffa2f95866de9711b0"
format_stamp: "Formatted at 2026-06-20 09:25:40 on dist-test-slave-414w"
server_key: "8e0ace69bfca59e36ea6b43baa67d529"
server_key_iv: "045ea47466895d8e0f07766565b9cbc2"
server_key_version: "encryptionkey@0"
I20260620 09:25:40.900696 25352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "e20954508c0a41ffa2f95866de9711b0"
format_stamp: "Formatted at 2026-06-20 09:25:40 on dist-test-slave-414w"
server_key: "8e0ace69bfca59e36ea6b43baa67d529"
server_key_iv: "045ea47466895d8e0f07766565b9cbc2"
server_key_version: "encryptionkey@0"
I20260620 09:25:40.902021 25352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:25:40.902777 25367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:40.902971 25352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:25:40.903064 25352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "e20954508c0a41ffa2f95866de9711b0"
format_stamp: "Formatted at 2026-06-20 09:25:40 on dist-test-slave-414w"
server_key: "8e0ace69bfca59e36ea6b43baa67d529"
server_key_iv: "045ea47466895d8e0f07766565b9cbc2"
server_key_version: "encryptionkey@0"
I20260620 09:25:40.903141 25352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:40.920693 25352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:40.922740 25352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:40.922896 25352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:40.926985 25352 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41829
I20260620 09:25:40.927019 25419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41829 every 8 connection(s)
I20260620 09:25:40.927457 25352 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:40.928419 25420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:40.930631 25420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0: Bootstrap starting.
I20260620 09:25:40.931141 25420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:40.931365 25420 log.cc:824] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:40.932060 25420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0: No bootstrap required, opened a new log
I20260620 09:25:40.932427 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 25352
I20260620 09:25:40.932516 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:40.932703 4332 external_mini_cluster.cc:1428] Setting key a420e44395e073c9448c9e11804dff03
I20260620 09:25:40.934306 25420 raft_consensus.cc:348] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20954508c0a41ffa2f95866de9711b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41829 } }
I20260620 09:25:40.934458 25420 raft_consensus.cc:374] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:40.934507 25420 raft_consensus.cc:729] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20954508c0a41ffa2f95866de9711b0, State: Initialized, Role: FOLLOWER
I20260620 09:25:40.934659 25420 consensus_queue.cc:260] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20954508c0a41ffa2f95866de9711b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41829 } }
I20260620 09:25:40.934757 25420 raft_consensus.cc:388] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:40.934801 25420 raft_consensus.cc:482] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:40.934849 25420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:40.935361 25420 raft_consensus.cc:504] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20954508c0a41ffa2f95866de9711b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41829 } }
I20260620 09:25:40.935478 25420 leader_election.cc:302] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e20954508c0a41ffa2f95866de9711b0; no voters:
I20260620 09:25:40.935650 25420 leader_election.cc:288] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:40.935683 25425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:40.935842 25425 raft_consensus.cc:686] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [term 1 LEADER]: Becoming Leader. State: Replica: e20954508c0a41ffa2f95866de9711b0, State: Running, Role: LEADER
I20260620 09:25:40.935894 25420 sys_catalog.cc:564] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:40.935988 25425 consensus_queue.cc:237] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20954508c0a41ffa2f95866de9711b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41829 } }
I20260620 09:25:40.936398 25426 sys_catalog.cc:455] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e20954508c0a41ffa2f95866de9711b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20954508c0a41ffa2f95866de9711b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41829 } } }
I20260620 09:25:40.936398 25427 sys_catalog.cc:455] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e20954508c0a41ffa2f95866de9711b0. Latest consensus state: current_term: 1 leader_uuid: "e20954508c0a41ffa2f95866de9711b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20954508c0a41ffa2f95866de9711b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41829 } } }
I20260620 09:25:40.936497 25426 sys_catalog.cc:458] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:40.936502 25427 sys_catalog.cc:458] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:40.936785 25434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:40.937816 25434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:40.939466 25424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:40.933222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.173:37619 (local address 127.0.0.1:41829)
0620 09:25:40.933406 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:40.933414 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:40.933442 (+ 28us) server_negotiation.cc:407] Connection header received
0620 09:25:40.933841 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:40.933847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:40.933891 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:40.934001 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:40.934232 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:40.935477 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:40.936647 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:40.937574 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:40.937716 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:40.938750 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:40.938753 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:40.938754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:40.938963 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:40.939002 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:40.939016 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:40.939127 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:40.939209 (+ 82us) server_negotiation.cc:299] Negotiation successful
0620 09:25:40.939238 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
I20260620 09:25:40.939472 25434 catalog_manager.cc:1347] Generated new cluster ID: 9f1cab89e09d4eaa802106ce96a22b8a
I20260620 09:25:40.939545 25434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:40.940327 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:40.948364 25424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:40.945440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.173:36829 (local address 127.0.0.1:41829)
0620 09:25:40.945479 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:40.945480 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:40.945684 (+ 204us) server_negotiation.cc:407] Connection header received
0620 09:25:40.945757 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:40.945758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:40.945767 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:40.945812 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:40.945968 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:40.946443 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:40.947172 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:40.947386 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:40.947467 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:40.948118 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:40.948120 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:40.948120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:40.948124 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:40.948137 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:40.948141 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:40.948189 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:40.948258 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 09:25:40.948267 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":10,"server-negotiator.queue_time_us":18}
I20260620 09:25:40.951282 25434 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:40.952005 25434 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:40.958703 25434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e20954508c0a41ffa2f95866de9711b0: Generated new TSK 0
I20260620 09:25:40.958874 25434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:40.967275 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 25352
W20260620 09:25:40.971490 25337 connection.cc:535] client connection to 127.0.0.1:41829 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41829): Connection reset by peer (error 104)
2026-06-20T09:25:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1302 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:41 dist-test-slave-414w krb5kdc[25451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:41 dist-test-slave-414w krb5kdc[25451](info): set up 2 sockets
Jun 20 09:25:41 dist-test-slave-414w krb5kdc[25451](info): commencing operation
krb5kdc: starting...
W20260620 09:25:42.001564 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:42 dist-test-slave-414w krb5kdc[25451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 09:25:42.113819 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39945
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:42.192739 25464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:42.192961 25464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:42.192984 25464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:42.194188 25464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:42.194232 25464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:42.194247 25464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:42.194259 25464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:42.194272 25464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:42.196918 25464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:42.197227 25464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:42.197471 25464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:42.202410 25469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:42.202410 25470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:42.202481 25464 server_base.cc:1034] running on GCE node
W20260620 09:25:42.202414 25472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:42.202850 25464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:42.203212 25464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:42.204380 25464 hybrid_clock.cc:648] HybridClock initialized: now 1781947542204358 us; error 39 us; skew 500 ppm
Jun 20 09:25:42 dist-test-slave-414w krb5kdc[25451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947542, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:42.206831 25464 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 09:25:42.207381 25464 webserver.cc:460] Webserver started at http://127.0.0.1:36219/ using document root <none> and password file <none>
I20260620 09:25:42.207607 25464 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:42.207674 25464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:42.207824 25464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:42.208874 25464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "9f5a7c03bc574968a7c1d965748eb307"
format_stamp: "Formatted at 2026-06-20 09:25:42 on dist-test-slave-414w"
server_key: "d7d9fc2d123fdd0754b9d234bddc6e0c"
server_key_iv: "2b9f648158a44f5a4ff59895491220c1"
server_key_version: "encryptionkey@0"
I20260620 09:25:42.209291 25464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "9f5a7c03bc574968a7c1d965748eb307"
format_stamp: "Formatted at 2026-06-20 09:25:42 on dist-test-slave-414w"
server_key: "d7d9fc2d123fdd0754b9d234bddc6e0c"
server_key_iv: "2b9f648158a44f5a4ff59895491220c1"
server_key_version: "encryptionkey@0"
I20260620 09:25:42.210584 25464 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:25:42.211360 25479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:42.211565 25464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:25:42.211661 25464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "9f5a7c03bc574968a7c1d965748eb307"
format_stamp: "Formatted at 2026-06-20 09:25:42 on dist-test-slave-414w"
server_key: "d7d9fc2d123fdd0754b9d234bddc6e0c"
server_key_iv: "2b9f648158a44f5a4ff59895491220c1"
server_key_version: "encryptionkey@0"
I20260620 09:25:42.211747 25464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:42.215039 25464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:42.216884 25464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:42.217046 25464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:42.221019 25464 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39945
I20260620 09:25:42.221086 25531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39945 every 8 connection(s)
I20260620 09:25:42.221416 25464 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:42.222395 25532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:42.224548 25532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307: Bootstrap starting.
I20260620 09:25:42.225112 25532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:42.225428 25532 log.cc:824] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:42.226145 25532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307: No bootstrap required, opened a new log
I20260620 09:25:42.227767 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 25464
I20260620 09:25:42.227713 25532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5a7c03bc574968a7c1d965748eb307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39945 } }
I20260620 09:25:42.227833 25532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:42.227898 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:42.227896 25532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f5a7c03bc574968a7c1d965748eb307, State: Initialized, Role: FOLLOWER
I20260620 09:25:42.228096 4332 external_mini_cluster.cc:1428] Setting key fdf3d6073815f72d7e93f81e97f64426
I20260620 09:25:42.228092 25532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5a7c03bc574968a7c1d965748eb307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39945 } }
I20260620 09:25:42.228226 25532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:42.228282 25532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:42.228343 25532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:42.229447 25532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5a7c03bc574968a7c1d965748eb307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39945 } }
I20260620 09:25:42.229552 25532 leader_election.cc:302] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f5a7c03bc574968a7c1d965748eb307; no voters:
I20260620 09:25:42.229730 25532 leader_election.cc:288] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:42.229787 25537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:42.229925 25537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [term 1 LEADER]: Becoming Leader. State: Replica: 9f5a7c03bc574968a7c1d965748eb307, State: Running, Role: LEADER
I20260620 09:25:42.230031 25532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:42.230054 25537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5a7c03bc574968a7c1d965748eb307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39945 } }
I20260620 09:25:42.230430 25539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f5a7c03bc574968a7c1d965748eb307" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5a7c03bc574968a7c1d965748eb307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39945 } } }
I20260620 09:25:42.230422 25537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f5a7c03bc574968a7c1d965748eb307. Latest consensus state: current_term: 1 leader_uuid: "9f5a7c03bc574968a7c1d965748eb307" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5a7c03bc574968a7c1d965748eb307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39945 } } }
I20260620 09:25:42.230458 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:42.228578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.173:41313 (local address 127.0.0.1:39945)
0620 09:25:42.228771 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:42.228780 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:42.228815 (+ 35us) server_negotiation.cc:407] Connection header received
0620 09:25:42.228955 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:42.228963 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:42.228999 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:42.229186 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:42.229858 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:42.229861 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:42.229862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:42.229866 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:42.229909 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:42.229923 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:42.229981 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:42.230223 (+ 242us) server_negotiation.cc:299] Negotiation successful
0620 09:25:42.230266 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":93,"threads_started":1}
I20260620 09:25:42.230525 25539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:42.230532 25537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:42.231323 25545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:42.231542 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:42.231868 25545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:42.233448 25545 catalog_manager.cc:1347] Generated new cluster ID: dcca5dd004b1414f8801440498ddc3c8
I20260620 09:25:42.233500 25545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:42.234058 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:42.232930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.173:33227 (local address 127.0.0.1:39945)
0620 09:25:42.232960 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:42.232962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:42.233094 (+ 132us) server_negotiation.cc:407] Connection header received
0620 09:25:42.233189 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:42.233190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:42.233195 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:42.233251 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:42.233830 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:42.233831 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:42.233831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:42.233836 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:42.233849 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:42.233853 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:42.233912 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:42.233965 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 09:25:42.233978 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 09:25:42.244796 25545 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:42.244962 25545 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:42.255385 25545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f5a7c03bc574968a7c1d965748eb307: Generated new TSK 0
I20260620 09:25:42.255569 25545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:42.263381 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 25464
2026-06-20T09:25:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1295 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:42 dist-test-slave-414w krb5kdc[25563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:42 dist-test-slave-414w krb5kdc[25563](info): set up 2 sockets
Jun 20 09:25:42 dist-test-slave-414w krb5kdc[25563](info): commencing operation
krb5kdc: starting...
W20260620 09:25:43.285456 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:43 dist-test-slave-414w krb5kdc[25563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 09:25:43.396991 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39699
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:43.476924 25576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:43.477126 25576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:43.477205 25576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:43.478363 25576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:43.478404 25576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:43.478420 25576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:43.478431 25576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:43.478444 25576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:43.479933 25576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39699
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:43.480180 25576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:43.480409 25576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:43.485466 25584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:43.485433 25581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:43.485998 25576 server_base.cc:1034] running on GCE node
W20260620 09:25:43.485468 25582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:43.486349 25576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:43.486675 25576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:43.487916 25576 hybrid_clock.cc:648] HybridClock initialized: now 1781947543487896 us; error 35 us; skew 500 ppm
Jun 20 09:25:43 dist-test-slave-414w krb5kdc[25563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947543, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:43.493999 25576 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 09:25:43.494457 25576 webserver.cc:460] Webserver started at http://127.0.0.1:46047/ using document root <none> and password file <none>
I20260620 09:25:43.494705 25576 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:43.494751 25576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:43.494907 25576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:43.495944 25576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "420ec0776c5a4c5b9ce5898fb4f97ce0"
format_stamp: "Formatted at 2026-06-20 09:25:43 on dist-test-slave-414w"
server_key: "1e778af2b297255fe73c082f90fa47d7"
server_key_iv: "3665b32c8000c31429537867a3186dc0"
server_key_version: "encryptionkey@0"
I20260620 09:25:43.496296 25576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "420ec0776c5a4c5b9ce5898fb4f97ce0"
format_stamp: "Formatted at 2026-06-20 09:25:43 on dist-test-slave-414w"
server_key: "1e778af2b297255fe73c082f90fa47d7"
server_key_iv: "3665b32c8000c31429537867a3186dc0"
server_key_version: "encryptionkey@0"
I20260620 09:25:43.497664 25576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:25:43.498412 25591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:43.498579 25576 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:25:43.498685 25576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "420ec0776c5a4c5b9ce5898fb4f97ce0"
format_stamp: "Formatted at 2026-06-20 09:25:43 on dist-test-slave-414w"
server_key: "1e778af2b297255fe73c082f90fa47d7"
server_key_iv: "3665b32c8000c31429537867a3186dc0"
server_key_version: "encryptionkey@0"
I20260620 09:25:43.498759 25576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:43.518658 25576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:43.520519 25576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:43.520674 25576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:43.524734 25576 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39699
I20260620 09:25:43.524782 25643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39699 every 8 connection(s)
I20260620 09:25:43.525130 25576 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:43.526105 25644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:43.528267 25644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0: Bootstrap starting.
I20260620 09:25:43.528789 25644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:43.529027 25644 log.cc:824] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:43.529645 25644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0: No bootstrap required, opened a new log
I20260620 09:25:43.531111 25644 raft_consensus.cc:348] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "420ec0776c5a4c5b9ce5898fb4f97ce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39699 } }
I20260620 09:25:43.531252 25644 raft_consensus.cc:374] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:43.531277 25644 raft_consensus.cc:729] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 420ec0776c5a4c5b9ce5898fb4f97ce0, State: Initialized, Role: FOLLOWER
I20260620 09:25:43.531417 25644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "420ec0776c5a4c5b9ce5898fb4f97ce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39699 } }
I20260620 09:25:43.531515 25644 raft_consensus.cc:388] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:43.531539 25644 raft_consensus.cc:482] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:43.531608 25644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:43.531647 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 25576
I20260620 09:25:43.531760 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:43.531941 4332 external_mini_cluster.cc:1428] Setting key 345da0d898bd0f75cd162205bad06dfd
I20260620 09:25:43.532289 25644 raft_consensus.cc:504] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "420ec0776c5a4c5b9ce5898fb4f97ce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39699 } }
I20260620 09:25:43.532407 25644 leader_election.cc:302] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 420ec0776c5a4c5b9ce5898fb4f97ce0; no voters:
I20260620 09:25:43.532575 25644 leader_election.cc:288] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:43.532599 25649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:43.532781 25649 raft_consensus.cc:686] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [term 1 LEADER]: Becoming Leader. State: Replica: 420ec0776c5a4c5b9ce5898fb4f97ce0, State: Running, Role: LEADER
I20260620 09:25:43.532905 25644 sys_catalog.cc:564] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:43.532908 25649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "420ec0776c5a4c5b9ce5898fb4f97ce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39699 } }
I20260620 09:25:43.533293 25649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 420ec0776c5a4c5b9ce5898fb4f97ce0. Latest consensus state: current_term: 1 leader_uuid: "420ec0776c5a4c5b9ce5898fb4f97ce0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "420ec0776c5a4c5b9ce5898fb4f97ce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39699 } } }
I20260620 09:25:43.533367 25649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:43.533444 25650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "420ec0776c5a4c5b9ce5898fb4f97ce0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "420ec0776c5a4c5b9ce5898fb4f97ce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39699 } } }
I20260620 09:25:43.533571 25650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:43.533743 25655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:43.534287 25655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:43.535861 25655 catalog_manager.cc:1347] Generated new cluster ID: f1d468ded72a4f229749476e47405dbf
I20260620 09:25:43.535912 25655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 09:25:43 dist-test-slave-414w krb5kdc[25563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 09:25:43.544651 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:43.532403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.173:44647 (local address 127.0.0.1:39699)
0620 09:25:43.532588 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:43.532598 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:43.532644 (+ 46us) server_negotiation.cc:407] Connection header received
0620 09:25:43.532757 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:43.532765 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:43.532889 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:43.533882 (+ 993us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:43.533994 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:43.535511 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:43.536575 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:43.537171 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:43.537265 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:43.540062 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:43.540066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:43.540066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:43.540078 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:43.542720 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:43.543167 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:43.543170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:43.543171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:43.543296 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:43.543528 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:43.543529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:43.543529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:43.543993 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:43.544206 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:43.544298 (+ 92us) server_negotiation.cc:299] Negotiation successful
0620 09:25:43.544415 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":88,"threads_started":1}
I20260620 09:25:43.545742 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:43.547640 25655 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:43.548367 25655 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:43.555140 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:43.547366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.173:55697 (local address 127.0.0.1:39699)
0620 09:25:43.547401 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:43.547402 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:43.547527 (+ 125us) server_negotiation.cc:407] Connection header received
0620 09:25:43.547629 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:43.547630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:43.547642 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:43.547712 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:43.548315 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:43.549206 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:43.550351 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:43.550664 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:43.550727 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:43.552209 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:43.552210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:43.552210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:43.552217 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:43.553888 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:43.554346 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:43.554346 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:43.554347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:43.554397 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:43.554638 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:43.554639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:43.554639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:43.554746 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:43.554831 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:43.555001 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 09:25:43.555057 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 09:25:43.557785 25655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 420ec0776c5a4c5b9ce5898fb4f97ce0: Generated new TSK 0
I20260620 09:25:43.557963 25655 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:43.563293 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 25576
W20260620 09:25:43.567539 25561 connection.cc:535] client connection to 127.0.0.1:39699 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39699): Connection reset by peer (error 104)
2026-06-20T09:25:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1299 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:43 dist-test-slave-414w krb5kdc[25675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:43 dist-test-slave-414w krb5kdc[25675](info): set up 2 sockets
Jun 20 09:25:43 dist-test-slave-414w krb5kdc[25675](info): commencing operation
krb5kdc: starting...
W20260620 09:25:44.601662 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:44 dist-test-slave-414w krb5kdc[25675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 09:25:44.716137 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42221
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:44.797640 25688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:44.797878 25688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:44.797914 25688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:44.799080 25688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:44.799153 25688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:44.799177 25688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:44.799204 25688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:44.799234 25688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:44.800693 25688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42221
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:44.800956 25688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:44.801210 25688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:44.805897 25696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:44.805878 25693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:44.805907 25694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:44.806229 25688 server_base.cc:1034] running on GCE node
I20260620 09:25:44.806458 25688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:44.806728 25688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:44.807879 25688 hybrid_clock.cc:648] HybridClock initialized: now 1781947544807871 us; error 32 us; skew 500 ppm
Jun 20 09:25:44 dist-test-slave-414w krb5kdc[25675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947544, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:44.810420 25688 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 09:25:44.810844 25688 webserver.cc:460] Webserver started at http://127.0.0.1:36241/ using document root <none> and password file <none>
I20260620 09:25:44.811082 25688 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:44.811154 25688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:44.811302 25688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:44.812321 25688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "34d8104c04cc4f6c857ef4c96991f04f"
format_stamp: "Formatted at 2026-06-20 09:25:44 on dist-test-slave-414w"
server_key: "7587ab54af2d9e7da4f24b5f1bd48694"
server_key_iv: "447b0d05c7c0718469968af629eef722"
server_key_version: "encryptionkey@0"
I20260620 09:25:44.812692 25688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "34d8104c04cc4f6c857ef4c96991f04f"
format_stamp: "Formatted at 2026-06-20 09:25:44 on dist-test-slave-414w"
server_key: "7587ab54af2d9e7da4f24b5f1bd48694"
server_key_iv: "447b0d05c7c0718469968af629eef722"
server_key_version: "encryptionkey@0"
I20260620 09:25:44.814074 25688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:25:44.814819 25703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:44.815011 25688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:25:44.815109 25688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "34d8104c04cc4f6c857ef4c96991f04f"
format_stamp: "Formatted at 2026-06-20 09:25:44 on dist-test-slave-414w"
server_key: "7587ab54af2d9e7da4f24b5f1bd48694"
server_key_iv: "447b0d05c7c0718469968af629eef722"
server_key_version: "encryptionkey@0"
I20260620 09:25:44.815191 25688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:44.828222 25688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:44.830355 25688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:44.830513 25688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:44.834350 25688 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42221
I20260620 09:25:44.834398 25755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42221 every 8 connection(s)
I20260620 09:25:44.834741 25688 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:44.835748 25756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:44.838016 25756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f: Bootstrap starting.
I20260620 09:25:44.838598 25756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:44.838882 25756 log.cc:824] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:44.839556 25756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f: No bootstrap required, opened a new log
I20260620 09:25:44.840416 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 25688
I20260620 09:25:44.840540 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:44.840706 4332 external_mini_cluster.cc:1428] Setting key 5fad817e8507b4578ed8617531feacbe
I20260620 09:25:44.841709 25756 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d8104c04cc4f6c857ef4c96991f04f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42221 } }
I20260620 09:25:44.841840 25756 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:44.841861 25756 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34d8104c04cc4f6c857ef4c96991f04f, State: Initialized, Role: FOLLOWER
I20260620 09:25:44.842011 25756 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d8104c04cc4f6c857ef4c96991f04f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42221 } }
I20260620 09:25:44.842100 25756 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:44.842136 25756 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:44.842195 25756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:44.843158 25756 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d8104c04cc4f6c857ef4c96991f04f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42221 } }
I20260620 09:25:44.843261 25756 leader_election.cc:302] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34d8104c04cc4f6c857ef4c96991f04f; no voters:
I20260620 09:25:44.843400 25756 leader_election.cc:288] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:44.843472 25761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:44.843614 25756 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:44.843621 25761 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [term 1 LEADER]: Becoming Leader. State: Replica: 34d8104c04cc4f6c857ef4c96991f04f, State: Running, Role: LEADER
I20260620 09:25:44.843827 25761 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d8104c04cc4f6c857ef4c96991f04f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42221 } }
I20260620 09:25:44.844143 25761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34d8104c04cc4f6c857ef4c96991f04f. Latest consensus state: current_term: 1 leader_uuid: "34d8104c04cc4f6c857ef4c96991f04f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d8104c04cc4f6c857ef4c96991f04f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42221 } } }
I20260620 09:25:44.844352 25761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:44.844280 25763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34d8104c04cc4f6c857ef4c96991f04f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d8104c04cc4f6c857ef4c96991f04f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42221 } } }
I20260620 09:25:44.844767 25763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f [sys.catalog]: This master's current role is: LEADER
W20260620 09:25:44.845420 25777 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:25:44.845476 25777 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:25:44.845530 25770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:44.845744 25770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:44.847221 25770 catalog_manager.cc:1347] Generated new cluster ID: e578d6372d0c4f70ac3c3037f8ce9606
I20260620 09:25:44.847329 25770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:44.847509 25760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:44.841222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.173:45239 (local address 127.0.0.1:42221)
0620 09:25:44.841409 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:44.841416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:44.841441 (+ 25us) server_negotiation.cc:407] Connection header received
0620 09:25:44.841726 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:44.841733 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:44.841787 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:44.841957 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:44.842213 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:44.843615 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:44.844797 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:44.845465 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:44.845589 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:44.846729 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:44.846732 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:44.846733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:44.846957 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:44.846993 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:44.847008 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:44.847087 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:44.847231 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 09:25:44.847259 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260620 09:25:44.848444 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:44.852630 25770 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:44.853120 25770 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:44.855811 25760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:44.850329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.173:35651 (local address 127.0.0.1:42221)
0620 09:25:44.850362 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:44.850364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:44.852349 (+ 1985us) server_negotiation.cc:407] Connection header received
0620 09:25:44.852376 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:44.852377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:44.852401 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:44.852483 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:44.852695 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:44.853472 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:44.854493 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:44.854813 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:44.854887 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:44.855558 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:44.855559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:44.855560 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:44.855564 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:44.855581 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:44.855586 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:44.855651 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:44.855725 (+ 74us) server_negotiation.cc:299] Negotiation successful
0620 09:25:44.855737 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 09:25:44.857338 25770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34d8104c04cc4f6c857ef4c96991f04f: Generated new TSK 0
I20260620 09:25:44.857502 25770 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:44.863345 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 25688
W20260620 09:25:44.867084 25673 connection.cc:535] client connection to 127.0.0.1:42221 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42221): Connection reset by peer (error 104)
2026-06-20T09:25:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1299 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:44 dist-test-slave-414w krb5kdc[25787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:44 dist-test-slave-414w krb5kdc[25787](info): set up 2 sockets
Jun 20 09:25:44 dist-test-slave-414w krb5kdc[25787](info): commencing operation
krb5kdc: starting...
W20260620 09:25:45.897450 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:45 dist-test-slave-414w krb5kdc[25787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 09:25:46.008173 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41187
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:46.086172 25800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:46.086354 25800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:46.086374 25800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:46.087489 25800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:46.087530 25800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:46.087546 25800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:46.087584 25800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:46.087610 25800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:46.089017 25800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41187
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:46.089287 25800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:46.089496 25800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:46.094055 25808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:46.094086 25806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:46.094105 25800 server_base.cc:1034] running on GCE node
W20260620 09:25:46.094144 25805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:46.094461 25800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:46.094743 25800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:46.095880 25800 hybrid_clock.cc:648] HybridClock initialized: now 1781947546095851 us; error 41 us; skew 500 ppm
Jun 20 09:25:46 dist-test-slave-414w krb5kdc[25787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947546, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:46.098364 25800 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 09:25:46.098770 25800 webserver.cc:460] Webserver started at http://127.0.0.1:34491/ using document root <none> and password file <none>
I20260620 09:25:46.098981 25800 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:46.099047 25800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:46.099193 25800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:46.100165 25800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "5d2b0667940c474f8707c9fe9754f303"
format_stamp: "Formatted at 2026-06-20 09:25:46 on dist-test-slave-414w"
server_key: "d86c831f16d2aac8e0d53bcd2f1b407b"
server_key_iv: "3047833c992d5ca01da7ce702a289108"
server_key_version: "encryptionkey@0"
I20260620 09:25:46.100507 25800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "5d2b0667940c474f8707c9fe9754f303"
format_stamp: "Formatted at 2026-06-20 09:25:46 on dist-test-slave-414w"
server_key: "d86c831f16d2aac8e0d53bcd2f1b407b"
server_key_iv: "3047833c992d5ca01da7ce702a289108"
server_key_version: "encryptionkey@0"
I20260620 09:25:46.101928 25800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:25:46.102596 25815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:46.102761 25800 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:46.102854 25800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "5d2b0667940c474f8707c9fe9754f303"
format_stamp: "Formatted at 2026-06-20 09:25:46 on dist-test-slave-414w"
server_key: "d86c831f16d2aac8e0d53bcd2f1b407b"
server_key_iv: "3047833c992d5ca01da7ce702a289108"
server_key_version: "encryptionkey@0"
I20260620 09:25:46.102931 25800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:46.106132 25800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:46.107831 25800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:46.107980 25800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:46.111677 25800 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41187
I20260620 09:25:46.111766 25867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41187 every 8 connection(s)
I20260620 09:25:46.112068 25800 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:46.113013 25868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:46.115151 25868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303: Bootstrap starting.
I20260620 09:25:46.115713 25868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:46.115994 25868 log.cc:824] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:46.116650 25868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303: No bootstrap required, opened a new log
I20260620 09:25:46.118252 25868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d2b0667940c474f8707c9fe9754f303" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41187 } }
I20260620 09:25:46.118384 25868 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:46.118405 25868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d2b0667940c474f8707c9fe9754f303, State: Initialized, Role: FOLLOWER
I20260620 09:25:46.118574 25868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d2b0667940c474f8707c9fe9754f303" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41187 } }
I20260620 09:25:46.118660 25868 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:46.118703 25868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:46.118750 25868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:46.119247 25868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d2b0667940c474f8707c9fe9754f303" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41187 } }
I20260620 09:25:46.119359 25868 leader_election.cc:302] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d2b0667940c474f8707c9fe9754f303; no voters:
I20260620 09:25:46.119539 25868 leader_election.cc:288] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:46.119599 25871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:46.119766 25871 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [term 1 LEADER]: Becoming Leader. State: Replica: 5d2b0667940c474f8707c9fe9754f303, State: Running, Role: LEADER
I20260620 09:25:46.119777 25868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:46.119867 25871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d2b0667940c474f8707c9fe9754f303" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41187 } }
I20260620 09:25:46.120209 25872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d2b0667940c474f8707c9fe9754f303" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d2b0667940c474f8707c9fe9754f303" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41187 } } }
I20260620 09:25:46.120302 25872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:46.120213 25873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d2b0667940c474f8707c9fe9754f303. Latest consensus state: current_term: 1 leader_uuid: "5d2b0667940c474f8707c9fe9754f303" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d2b0667940c474f8707c9fe9754f303" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41187 } } }
I20260620 09:25:46.120425 25873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:46.120508 25875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:46.120994 25875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:46.122054 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 25800
I20260620 09:25:46.122190 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:46.122370 4332 external_mini_cluster.cc:1428] Setting key f246a9353cf880e2caff11e705316a51
I20260620 09:25:46.122495 25875 catalog_manager.cc:1347] Generated new cluster ID: c152f3c98bbe4caaa4cb4140d30299e7
I20260620 09:25:46.122548 25875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:46.124575 25891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:46.122982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.173:52041 (local address 127.0.0.1:41187)
0620 09:25:46.123131 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:46.123140 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:46.123170 (+ 30us) server_negotiation.cc:407] Connection header received
0620 09:25:46.123268 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:46.123272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:46.123294 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:46.123427 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:46.124101 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:46.124104 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:46.124105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:46.124109 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:46.124161 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:46.124172 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:46.124245 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:46.124317 (+ 72us) server_negotiation.cc:299] Negotiation successful
0620 09:25:46.124445 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260620 09:25:46.125422 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:46.134326 25891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:46.126747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.173:42057 (local address 127.0.0.1:41187)
0620 09:25:46.126785 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:46.126786 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:46.133347 (+ 6561us) server_negotiation.cc:407] Connection header received
0620 09:25:46.133421 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:46.133422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:46.133428 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:46.133483 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:46.134083 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:46.134085 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:46.134085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:46.134090 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:46.134103 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:46.134107 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:46.134171 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:46.134227 (+ 56us) server_negotiation.cc:299] Negotiation successful
0620 09:25:46.134241 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 09:25:46.145560 25875 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:46.145743 25875 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:46.150307 25875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d2b0667940c474f8707c9fe9754f303: Generated new TSK 0
I20260620 09:25:46.150488 25875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:46.160141 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 25800
2026-06-20T09:25:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:46 dist-test-slave-414w krb5kdc[25899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:46 dist-test-slave-414w krb5kdc[25899](info): set up 2 sockets
Jun 20 09:25:46 dist-test-slave-414w krb5kdc[25899](info): commencing operation
krb5kdc: starting...
W20260620 09:25:47.209937 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:47 dist-test-slave-414w krb5kdc[25899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:25:47.322302 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:33913
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:34787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:33913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:47.400446 25912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:47.400632 25912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:47.400655 25912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:47.401726 25912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:47.401760 25912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:47.401774 25912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:47.401787 25912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:47.401799 25912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:47.403153 25912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:34787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:33913
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:33913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:47.403326 25912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:47.403493 25912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:47.408244 25918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:47.408238 25912 server_base.cc:1034] running on GCE node
W20260620 09:25:47.408255 25920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:47.408238 25917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:47.408681 25912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:47.408913 25912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:47.410058 25912 hybrid_clock.cc:648] HybridClock initialized: now 1781947547410042 us; error 32 us; skew 500 ppm
Jun 20 09:25:47 dist-test-slave-414w krb5kdc[25899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947547, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:47.412554 25912 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:25:47.413007 25912 webserver.cc:460] Webserver started at http://127.4.59.62:44427/ using document root <none> and password file <none>
I20260620 09:25:47.413281 25912 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:47.413345 25912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:47.413494 25912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:47.414506 25912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "ae1ad8b155d442449e0604db14a45f7e"
format_stamp: "Formatted at 2026-06-20 09:25:47 on dist-test-slave-414w"
server_key: "77856ec1153dc1407b65bbdd808c4b82"
server_key_iv: "2eca323d0f152a3406f1bb6ad1210347"
server_key_version: "encryptionkey@0"
I20260620 09:25:47.414860 25912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "ae1ad8b155d442449e0604db14a45f7e"
format_stamp: "Formatted at 2026-06-20 09:25:47 on dist-test-slave-414w"
server_key: "77856ec1153dc1407b65bbdd808c4b82"
server_key_iv: "2eca323d0f152a3406f1bb6ad1210347"
server_key_version: "encryptionkey@0"
I20260620 09:25:47.416198 25912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:25:47.416941 25927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:47.417133 25912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:47.417277 25912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "ae1ad8b155d442449e0604db14a45f7e"
format_stamp: "Formatted at 2026-06-20 09:25:47 on dist-test-slave-414w"
server_key: "77856ec1153dc1407b65bbdd808c4b82"
server_key_iv: "2eca323d0f152a3406f1bb6ad1210347"
server_key_version: "encryptionkey@0"
I20260620 09:25:47.417359 25912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:47.432789 25912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:47.435072 25912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:47.435232 25912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:47.439339 25912 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:33913
I20260620 09:25:47.439374 25979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:33913 every 8 connection(s)
I20260620 09:25:47.439723 25912 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:47.440503 25980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:47.442724 25980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e: Bootstrap starting.
I20260620 09:25:47.443287 25980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:47.443565 25980 log.cc:824] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:47.444206 25980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e: No bootstrap required, opened a new log
I20260620 09:25:47.445730 25980 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1ad8b155d442449e0604db14a45f7e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33913 } }
I20260620 09:25:47.445858 25980 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:47.445879 25980 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae1ad8b155d442449e0604db14a45f7e, State: Initialized, Role: FOLLOWER
I20260620 09:25:47.446062 25980 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1ad8b155d442449e0604db14a45f7e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33913 } }
I20260620 09:25:47.446125 25980 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:47.446146 25980 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:47.446210 25980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:47.446424 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 25912
I20260620 09:25:47.446563 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:47.446756 4332 external_mini_cluster.cc:1428] Setting key 5daf44eb3f17eb6a514f91f7aaa661a8
I20260620 09:25:47.447258 25980 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1ad8b155d442449e0604db14a45f7e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33913 } }
I20260620 09:25:47.447422 25980 leader_election.cc:302] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae1ad8b155d442449e0604db14a45f7e; no voters:
I20260620 09:25:47.447567 25980 leader_election.cc:288] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:47.447784 25980 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:47.447885 25986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:47.448011 25986 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [term 1 LEADER]: Becoming Leader. State: Replica: ae1ad8b155d442449e0604db14a45f7e, State: Running, Role: LEADER
I20260620 09:25:47.448148 25986 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1ad8b155d442449e0604db14a45f7e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33913 } }
I20260620 09:25:47.448559 25987 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae1ad8b155d442449e0604db14a45f7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1ad8b155d442449e0604db14a45f7e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33913 } } }
I20260620 09:25:47.448644 25987 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:47.448776 25987 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae1ad8b155d442449e0604db14a45f7e. Latest consensus state: current_term: 1 leader_uuid: "ae1ad8b155d442449e0604db14a45f7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1ad8b155d442449e0604db14a45f7e" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 33913 } } }
I20260620 09:25:47.448837 25987 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:47.449028 25995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:47.449669 25995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:47.451112 25995 catalog_manager.cc:1347] Generated new cluster ID: 1b0d4d225c3843a7b79a82b7d5345286
I20260620 09:25:47.451162 25995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 09:25:47 dist-test-slave-414w krb5kdc[25899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:47.458082 25984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:47.447190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45643 (local address 127.4.59.62:33913)
0620 09:25:47.447390 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:47.447399 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:47.447441 (+ 42us) server_negotiation.cc:407] Connection header received
0620 09:25:47.447554 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:47.447561 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:47.447624 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:47.447770 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:47.448511 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:47.449992 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:47.451055 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:47.451638 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:47.451756 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:47.454329 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:47.454333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:47.454334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:47.454345 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:47.456407 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:47.456826 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:47.456829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:47.456829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:47.456945 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:47.457197 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:47.457197 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:47.457198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:47.457511 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:47.457692 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:47.457776 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 09:25:47.457877 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
I20260620 09:25:47.459026 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:47.467294 25984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:47.460414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56379 (local address 127.4.59.62:33913)
0620 09:25:47.460453 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:47.460455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:47.460584 (+ 129us) server_negotiation.cc:407] Connection header received
0620 09:25:47.460614 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:47.460615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:47.460624 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:47.460686 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:47.461251 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:47.461991 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:47.462921 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:47.463269 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:47.463350 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:47.464586 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:47.464587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:47.464588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:47.464594 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:47.466145 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:47.466540 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:47.466541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:47.466541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:47.466597 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:47.466810 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:47.466811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:47.466811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:47.466934 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:47.467051 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:47.467143 (+ 92us) server_negotiation.cc:299] Negotiation successful
0620 09:25:47.467207 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 09:25:47.477133 25995 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:47.477833 25995 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:47.483523 25995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae1ad8b155d442449e0604db14a45f7e: Generated new TSK 0
I20260620 09:25:47.483688 25995 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:47.487193 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 25912
W20260620 09:25:47.491243 25897 connection.cc:535] client connection to 127.4.59.62:33913 recv error: Network error: failed to read from TLS socket (remote: 127.4.59.62:33913): Connection reset by peer (error 104)
2026-06-20T09:25:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1326 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:47 dist-test-slave-414w krb5kdc[26011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:47 dist-test-slave-414w krb5kdc[26011](info): set up 2 sockets
Jun 20 09:25:47 dist-test-slave-414w krb5kdc[26011](info): commencing operation
krb5kdc: starting...
W20260620 09:25:48.522261 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:48 dist-test-slave-414w krb5kdc[26011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:25:48.636056 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:40921
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:34355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:40921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:48.713238 26024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:48.713454 26024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:48.713490 26024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:48.714601 26024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:48.714666 26024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:48.714690 26024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:48.714723 26024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:48.714756 26024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:48.716163 26024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:34355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:40921
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:40921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:48.716425 26024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:48.716638 26024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:48.721287 26029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:48.721316 26032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:48.721316 26030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:48.721457 26024 server_base.cc:1034] running on GCE node
I20260620 09:25:48.721766 26024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:48.722167 26024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:48.723348 26024 hybrid_clock.cc:648] HybridClock initialized: now 1781947548723324 us; error 58 us; skew 500 ppm
Jun 20 09:25:48 dist-test-slave-414w krb5kdc[26011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947548, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:48.739771 26024 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:25:48.740252 26024 webserver.cc:460] Webserver started at http://127.4.59.62:43509/ using document root <none> and password file <none>
I20260620 09:25:48.740447 26024 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:48.740487 26024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:48.740577 26024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:48.741766 26024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "6cfce9169042447a884725ec93deb12c"
format_stamp: "Formatted at 2026-06-20 09:25:48 on dist-test-slave-414w"
server_key: "d50297e8f07c7c69914713e4794c678e"
server_key_iv: "3f8c5f379e79b60b0f762da8132debfd"
server_key_version: "encryptionkey@0"
I20260620 09:25:48.742247 26024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "6cfce9169042447a884725ec93deb12c"
format_stamp: "Formatted at 2026-06-20 09:25:48 on dist-test-slave-414w"
server_key: "d50297e8f07c7c69914713e4794c678e"
server_key_iv: "3f8c5f379e79b60b0f762da8132debfd"
server_key_version: "encryptionkey@0"
I20260620 09:25:48.743665 26024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:25:48.744402 26039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:48.744702 26024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:48.744810 26024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "6cfce9169042447a884725ec93deb12c"
format_stamp: "Formatted at 2026-06-20 09:25:48 on dist-test-slave-414w"
server_key: "d50297e8f07c7c69914713e4794c678e"
server_key_iv: "3f8c5f379e79b60b0f762da8132debfd"
server_key_version: "encryptionkey@0"
I20260620 09:25:48.744886 26024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:48.777863 26024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:48.779711 26024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:48.779881 26024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:48.783880 26024 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:40921
I20260620 09:25:48.783911 26091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:40921 every 8 connection(s)
I20260620 09:25:48.784250 26024 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:48.785045 26092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:48.787235 26092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c: Bootstrap starting.
I20260620 09:25:48.787748 26092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:48.787967 26092 log.cc:824] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:48.788573 26092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c: No bootstrap required, opened a new log
I20260620 09:25:48.790144 26092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cfce9169042447a884725ec93deb12c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40921 } }
I20260620 09:25:48.790277 26092 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:48.790300 26092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfce9169042447a884725ec93deb12c, State: Initialized, Role: FOLLOWER
I20260620 09:25:48.790428 26092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cfce9169042447a884725ec93deb12c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40921 } }
I20260620 09:25:48.790505 26092 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:48.790527 26092 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:48.790555 26092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:48.790860 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 26024
I20260620 09:25:48.790974 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:48.791200 4332 external_mini_cluster.cc:1428] Setting key ff28bdc2da565643bb6d39ce53664da4
I20260620 09:25:48.791643 26092 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cfce9169042447a884725ec93deb12c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40921 } }
I20260620 09:25:48.791774 26092 leader_election.cc:302] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cfce9169042447a884725ec93deb12c; no voters:
I20260620 09:25:48.792068 26092 leader_election.cc:288] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:48.792093 26097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:48.792269 26097 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [term 1 LEADER]: Becoming Leader. State: Replica: 6cfce9169042447a884725ec93deb12c, State: Running, Role: LEADER
I20260620 09:25:48.792351 26092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:48.792431 26097 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cfce9169042447a884725ec93deb12c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40921 } }
I20260620 09:25:48.792798 26097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cfce9169042447a884725ec93deb12c. Latest consensus state: current_term: 1 leader_uuid: "6cfce9169042447a884725ec93deb12c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cfce9169042447a884725ec93deb12c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40921 } } }
I20260620 09:25:48.792883 26097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:48.792810 26098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cfce9169042447a884725ec93deb12c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cfce9169042447a884725ec93deb12c" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 40921 } } }
I20260620 09:25:48.793078 26098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:48.793071 26105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:48.793690 26105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:48.795306 26105 catalog_manager.cc:1347] Generated new cluster ID: 8308b11420894f44a341610077b2f9c5
I20260620 09:25:48.795372 26105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:48.797603 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:48.791715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47985 (local address 127.4.59.62:40921)
0620 09:25:48.791889 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:48.791905 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:48.791960 (+ 55us) server_negotiation.cc:407] Connection header received
0620 09:25:48.792092 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:48.792099 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:48.792156 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:48.792586 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:48.792678 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:48.794270 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:48.795248 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:48.795822 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:48.795928 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:48.796817 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:48.796821 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:48.796822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:48.797025 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:48.797061 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:48.797075 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:48.797174 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:48.797371 (+ 197us) server_negotiation.cc:299] Negotiation successful
0620 09:25:48.797398 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260620 09:25:48.798592 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:48.809100 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:48.805446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37425 (local address 127.4.59.62:40921)
0620 09:25:48.805485 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:48.805488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:48.805598 (+ 110us) server_negotiation.cc:407] Connection header received
0620 09:25:48.805645 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:48.805645 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:48.805664 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:48.805732 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:48.805893 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:48.806688 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:48.807716 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:48.808033 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:48.808105 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:48.808836 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:48.808837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:48.808837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:48.808842 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:48.808856 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:48.808861 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:48.808924 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:48.808994 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 09:25:48.809008 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:25:48.816956 26105 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:48.817574 26105 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:48.825243 26105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cfce9169042447a884725ec93deb12c: Generated new TSK 0
I20260620 09:25:48.825472 26105 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:48.829957 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 26024
W20260620 09:25:48.834569 26009 connection.cc:535] client connection to 127.4.59.62:40921 recv error: Network error: failed to read from TLS socket (remote: 127.4.59.62:40921): Connection reset by peer (error 104)
2026-06-20T09:25:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1343 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:48 dist-test-slave-414w krb5kdc[26123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:48 dist-test-slave-414w krb5kdc[26123](info): set up 2 sockets
Jun 20 09:25:48 dist-test-slave-414w krb5kdc[26123](info): commencing operation
krb5kdc: starting...
W20260620 09:25:49.861471 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:49 dist-test-slave-414w krb5kdc[26123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:25:49.983911 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:39619
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:39619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:50.066918 26136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:50.067104 26136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:50.067127 26136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:50.068344 26136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:50.068383 26136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:50.068398 26136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:50.068411 26136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:50.068423 26136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:50.069888 26136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:39619
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:39619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:50.070133 26136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:50.070362 26136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:50.075210 26141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:50.075184 26144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:50.075194 26142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:50.075333 26136 server_base.cc:1034] running on GCE node
I20260620 09:25:50.075621 26136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:50.075861 26136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:50.077005 26136 hybrid_clock.cc:648] HybridClock initialized: now 1781947550076992 us; error 28 us; skew 500 ppm
Jun 20 09:25:50 dist-test-slave-414w krb5kdc[26123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947550, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:50.079442 26136 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:25:50.079849 26136 webserver.cc:460] Webserver started at http://127.4.59.62:42409/ using document root <none> and password file <none>
I20260620 09:25:50.080080 26136 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:50.080150 26136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:50.080298 26136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:50.081377 26136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "c5685c5066e2461d8ad3d6b30456da09"
format_stamp: "Formatted at 2026-06-20 09:25:50 on dist-test-slave-414w"
server_key: "c261c7c9eeb269bc23c5285e666e817f"
server_key_iv: "5f9cb9609bb286bf2211880d07307104"
server_key_version: "encryptionkey@0"
I20260620 09:25:50.081730 26136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "c5685c5066e2461d8ad3d6b30456da09"
format_stamp: "Formatted at 2026-06-20 09:25:50 on dist-test-slave-414w"
server_key: "c261c7c9eeb269bc23c5285e666e817f"
server_key_iv: "5f9cb9609bb286bf2211880d07307104"
server_key_version: "encryptionkey@0"
I20260620 09:25:50.082998 26136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260620 09:25:50.083693 26151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:50.083884 26136 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:50.083971 26136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "c5685c5066e2461d8ad3d6b30456da09"
format_stamp: "Formatted at 2026-06-20 09:25:50 on dist-test-slave-414w"
server_key: "c261c7c9eeb269bc23c5285e666e817f"
server_key_iv: "5f9cb9609bb286bf2211880d07307104"
server_key_version: "encryptionkey@0"
I20260620 09:25:50.084050 26136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:50.087278 26136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:50.089020 26136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:50.089203 26136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:50.092772 26136 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:39619
I20260620 09:25:50.092803 26203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:39619 every 8 connection(s)
I20260620 09:25:50.093116 26136 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:50.094110 26204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:50.096222 26204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09: Bootstrap starting.
I20260620 09:25:50.096764 26204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:50.097034 26204 log.cc:824] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:50.097703 26204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09: No bootstrap required, opened a new log
I20260620 09:25:50.098690 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 26136
I20260620 09:25:50.098810 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:50.098980 4332 external_mini_cluster.cc:1428] Setting key e84bede3c498439609ef02744c44ab55
I20260620 09:25:50.100149 26204 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5685c5066e2461d8ad3d6b30456da09" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39619 } }
I20260620 09:25:50.100324 26204 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:50.100363 26204 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5685c5066e2461d8ad3d6b30456da09, State: Initialized, Role: FOLLOWER
I20260620 09:25:50.100536 26204 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5685c5066e2461d8ad3d6b30456da09" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39619 } }
I20260620 09:25:50.100651 26204 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:50.100695 26204 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:50.100737 26204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:50.101343 26208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:50.099433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48949 (local address 127.4.59.62:39619)
0620 09:25:50.099667 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:50.099676 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:50.099725 (+ 49us) server_negotiation.cc:407] Connection header received
0620 09:25:50.100112 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:50.100120 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:50.100152 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:50.100299 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:50.100901 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:50.100905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:50.100905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:50.100910 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:50.100953 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:50.100967 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:50.101042 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:50.101094 (+ 52us) server_negotiation.cc:299] Negotiation successful
0620 09:25:50.101132 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":110,"threads_started":1}
I20260620 09:25:50.101454 26204 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5685c5066e2461d8ad3d6b30456da09" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39619 } }
I20260620 09:25:50.101634 26204 leader_election.cc:302] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5685c5066e2461d8ad3d6b30456da09; no voters:
I20260620 09:25:50.101795 26204 leader_election.cc:288] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:50.101886 26209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:50.102028 26204 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:50.102042 26209 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [term 1 LEADER]: Becoming Leader. State: Replica: c5685c5066e2461d8ad3d6b30456da09, State: Running, Role: LEADER
I20260620 09:25:50.102228 26209 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5685c5066e2461d8ad3d6b30456da09" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39619 } }
I20260620 09:25:50.102433 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:50.102546 26211 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5685c5066e2461d8ad3d6b30456da09. Latest consensus state: current_term: 1 leader_uuid: "c5685c5066e2461d8ad3d6b30456da09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5685c5066e2461d8ad3d6b30456da09" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39619 } } }
I20260620 09:25:50.102551 26210 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5685c5066e2461d8ad3d6b30456da09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5685c5066e2461d8ad3d6b30456da09" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39619 } } }
I20260620 09:25:50.102625 26210 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:50.102624 26211 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:50.103369 26218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:50.104050 26218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:50.105494 26218 catalog_manager.cc:1347] Generated new cluster ID: da0b709ed13b4a94a8a3a64bf992f1a4
I20260620 09:25:50.105546 26218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:50.105861 26208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:50.104719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57155 (local address 127.4.59.62:39619)
0620 09:25:50.104752 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:50.104754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:50.104857 (+ 103us) server_negotiation.cc:407] Connection header received
0620 09:25:50.104948 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:50.104948 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:50.104953 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:50.105014 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:50.105646 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:50.105647 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:50.105648 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:50.105653 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:50.105667 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:50.105671 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:50.105732 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:50.105797 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 09:25:50.105809 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 09:25:50.122749 26218 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:50.122920 26218 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:50.126760 26218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5685c5066e2461d8ad3d6b30456da09: Generated new TSK 0
I20260620 09:25:50.126933 26218 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:50.131080 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 26136
2026-06-20T09:25:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:50 dist-test-slave-414w krb5kdc[26235](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:50 dist-test-slave-414w krb5kdc[26235](info): set up 2 sockets
Jun 20 09:25:50 dist-test-slave-414w krb5kdc[26235](info): commencing operation
krb5kdc: starting...
W20260620 09:25:51.161486 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:51 dist-test-slave-414w krb5kdc[26235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:25:51.275807 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:39515
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:39515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:51.353636 26248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:51.353853 26248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:51.353888 26248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:51.355018 26248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:51.355085 26248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:51.355110 26248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:51.355154 26248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:51.355176 26248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:51.356585 26248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:39515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:39515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:51.356848 26248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:51.357069 26248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:51.361596 26253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:51.361743 26248 server_base.cc:1034] running on GCE node
W20260620 09:25:51.361608 26256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:51.361608 26254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:51.362059 26248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:51.362332 26248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:51.363489 26248 hybrid_clock.cc:648] HybridClock initialized: now 1781947551363481 us; error 36 us; skew 500 ppm
Jun 20 09:25:51 dist-test-slave-414w krb5kdc[26235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947551, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:51.365890 26248 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:25:51.366333 26248 webserver.cc:460] Webserver started at http://127.4.59.62:46719/ using document root <none> and password file <none>
I20260620 09:25:51.366541 26248 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:51.366605 26248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:51.366755 26248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:51.367748 26248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "d3b0cd6d7558419fbe098cafdc3e83b6"
format_stamp: "Formatted at 2026-06-20 09:25:51 on dist-test-slave-414w"
server_key: "5d9670a0c63d70fcd4160778f60ee9d6"
server_key_iv: "a912ac53981a98e045fe84466dd8a14f"
server_key_version: "encryptionkey@0"
I20260620 09:25:51.368088 26248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "d3b0cd6d7558419fbe098cafdc3e83b6"
format_stamp: "Formatted at 2026-06-20 09:25:51 on dist-test-slave-414w"
server_key: "5d9670a0c63d70fcd4160778f60ee9d6"
server_key_iv: "a912ac53981a98e045fe84466dd8a14f"
server_key_version: "encryptionkey@0"
I20260620 09:25:51.369405 26248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:25:51.370102 26263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:51.370266 26248 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:51.370369 26248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "d3b0cd6d7558419fbe098cafdc3e83b6"
format_stamp: "Formatted at 2026-06-20 09:25:51 on dist-test-slave-414w"
server_key: "5d9670a0c63d70fcd4160778f60ee9d6"
server_key_iv: "a912ac53981a98e045fe84466dd8a14f"
server_key_version: "encryptionkey@0"
I20260620 09:25:51.370440 26248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:51.378803 26248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:51.380968 26248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:51.381125 26248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:51.384773 26248 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:39515
I20260620 09:25:51.385222 26248 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:51.386137 26316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:51.386718 26315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:39515 every 8 connection(s)
I20260620 09:25:51.388441 26316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6: Bootstrap starting.
I20260620 09:25:51.389006 26316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:51.389365 26316 log.cc:824] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:51.389652 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 26248
I20260620 09:25:51.389765 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:51.389938 4332 external_mini_cluster.cc:1428] Setting key 77bc5a8aec175ad6fe3c2d52dc24c3fc
I20260620 09:25:51.390149 26316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6: No bootstrap required, opened a new log
I20260620 09:25:51.391885 26316 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b0cd6d7558419fbe098cafdc3e83b6" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39515 } }
I20260620 09:25:51.392050 26316 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:51.392112 26316 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3b0cd6d7558419fbe098cafdc3e83b6, State: Initialized, Role: FOLLOWER
I20260620 09:25:51.392241 26316 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b0cd6d7558419fbe098cafdc3e83b6" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39515 } }
I20260620 09:25:51.392335 26316 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:51.392380 26316 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:51.392426 26316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:51.393268 26316 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b0cd6d7558419fbe098cafdc3e83b6" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39515 } }
I20260620 09:25:51.393380 26316 leader_election.cc:302] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3b0cd6d7558419fbe098cafdc3e83b6; no voters:
I20260620 09:25:51.393532 26316 leader_election.cc:288] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:51.393576 26321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:51.393710 26321 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [term 1 LEADER]: Becoming Leader. State: Replica: d3b0cd6d7558419fbe098cafdc3e83b6, State: Running, Role: LEADER
I20260620 09:25:51.393778 26316 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:51.393824 26321 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b0cd6d7558419fbe098cafdc3e83b6" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39515 } }
I20260620 09:25:51.394132 26321 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3b0cd6d7558419fbe098cafdc3e83b6. Latest consensus state: current_term: 1 leader_uuid: "d3b0cd6d7558419fbe098cafdc3e83b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b0cd6d7558419fbe098cafdc3e83b6" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39515 } } }
I20260620 09:25:51.394152 26322 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3b0cd6d7558419fbe098cafdc3e83b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b0cd6d7558419fbe098cafdc3e83b6" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 39515 } } }
I20260620 09:25:51.394272 26321 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:51.394486 26322 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:51.394515 26329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:51.395047 26329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:51.396574 26329 catalog_manager.cc:1347] Generated new cluster ID: 0d10243328464f82b34253db783cf522
I20260620 09:25:51.396636 26329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 09:25:51 dist-test-slave-414w krb5kdc[26235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:51.402323 26320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:51.390459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53653 (local address 127.4.59.62:39515)
0620 09:25:51.390635 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:51.390644 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:51.390676 (+ 32us) server_negotiation.cc:407] Connection header received
0620 09:25:51.391879 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:51.391887 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:51.391941 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:51.392128 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:51.392857 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:51.394121 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:51.395268 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:51.395850 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:51.395957 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:51.398580 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:51.398585 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:51.398585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:51.398598 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:51.400596 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:51.400976 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:51.400979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:51.400979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:51.401122 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:51.401369 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:51.401370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:51.401370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:51.401674 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:51.401862 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:51.401949 (+ 87us) server_negotiation.cc:299] Negotiation successful
0620 09:25:51.402127 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":100,"threads_started":1}
I20260620 09:25:51.403357 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:51.411612 26320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:51.404706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45793 (local address 127.4.59.62:39515)
0620 09:25:51.404775 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:51.404777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:51.404890 (+ 113us) server_negotiation.cc:407] Connection header received
0620 09:25:51.404945 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:51.404946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:51.404964 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:51.405018 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:51.405649 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:51.406345 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:51.407324 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:51.407616 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:51.407673 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:51.408951 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:51.408953 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:51.408953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:51.408974 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:51.410504 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:51.410883 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:51.410884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:51.410884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:51.410934 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:51.411141 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:51.411141 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:51.411142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:51.411251 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:51.411343 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:51.411461 (+ 118us) server_negotiation.cc:299] Negotiation successful
0620 09:25:51.411526 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 09:25:51.414137 26329 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:51.414601 26329 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:51.421609 26329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3b0cd6d7558419fbe098cafdc3e83b6: Generated new TSK 0
I20260620 09:25:51.421801 26329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:51.429250 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 26248
2026-06-20T09:25:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:51 dist-test-slave-414w krb5kdc[26347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:51 dist-test-slave-414w krb5kdc[26347](info): set up 2 sockets
Jun 20 09:25:51 dist-test-slave-414w krb5kdc[26347](info): commencing operation
krb5kdc: starting...
W20260620 09:25:52.457636 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:52 dist-test-slave-414w krb5kdc[26347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:25:52.569828 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:34671
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:39313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:34671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:52.648191 26360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:52.648398 26360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:52.648423 26360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:52.649521 26360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:52.649554 26360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:52.649569 26360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:52.649581 26360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:52.649595 26360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:52.650951 26360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:39313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:34671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:34671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:52.651132 26360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:52.651283 26360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:52.655944 26366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:52.655992 26368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:52.656038 26360 server_base.cc:1034] running on GCE node
W20260620 09:25:52.656003 26365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:52.656350 26360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:52.656572 26360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:52.657712 26360 hybrid_clock.cc:648] HybridClock initialized: now 1781947552657701 us; error 29 us; skew 500 ppm
Jun 20 09:25:52 dist-test-slave-414w krb5kdc[26347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947552, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:52.660149 26360 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:25:52.660619 26360 webserver.cc:460] Webserver started at http://127.4.59.62:40833/ using document root <none> and password file <none>
I20260620 09:25:52.660856 26360 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:52.660952 26360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:52.661105 26360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:52.662132 26360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "e39979f3981c4607a05adce3726ca67a"
format_stamp: "Formatted at 2026-06-20 09:25:52 on dist-test-slave-414w"
server_key: "3e2d2a789f719c7d2b532465d9d6453d"
server_key_iv: "67cd877a45a8c9f1690eb54c32b0025d"
server_key_version: "encryptionkey@0"
I20260620 09:25:52.662487 26360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "e39979f3981c4607a05adce3726ca67a"
format_stamp: "Formatted at 2026-06-20 09:25:52 on dist-test-slave-414w"
server_key: "3e2d2a789f719c7d2b532465d9d6453d"
server_key_iv: "67cd877a45a8c9f1690eb54c32b0025d"
server_key_version: "encryptionkey@0"
I20260620 09:25:52.664244 26360 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260620 09:25:52.664963 26375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:52.665191 26360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:52.665300 26360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "e39979f3981c4607a05adce3726ca67a"
format_stamp: "Formatted at 2026-06-20 09:25:52 on dist-test-slave-414w"
server_key: "3e2d2a789f719c7d2b532465d9d6453d"
server_key_iv: "67cd877a45a8c9f1690eb54c32b0025d"
server_key_version: "encryptionkey@0"
I20260620 09:25:52.665388 26360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:52.692027 26360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:52.693908 26360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:52.694078 26360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:52.698091 26360 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:34671
I20260620 09:25:52.698143 26427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:34671 every 8 connection(s)
I20260620 09:25:52.698508 26360 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:52.699378 26428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:52.701568 26428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a: Bootstrap starting.
I20260620 09:25:52.702131 26428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:52.702420 26428 log.cc:824] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:52.703102 26428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a: No bootstrap required, opened a new log
I20260620 09:25:52.704195 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 26360
I20260620 09:25:52.704303 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:52.704487 4332 external_mini_cluster.cc:1428] Setting key 14070052b55bb65701790e4ff3fc6f17
I20260620 09:25:52.705199 26428 raft_consensus.cc:348] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e39979f3981c4607a05adce3726ca67a" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 34671 } }
I20260620 09:25:52.705368 26428 raft_consensus.cc:374] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:52.705415 26428 raft_consensus.cc:729] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e39979f3981c4607a05adce3726ca67a, State: Initialized, Role: FOLLOWER
I20260620 09:25:52.705539 26428 consensus_queue.cc:260] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e39979f3981c4607a05adce3726ca67a" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 34671 } }
I20260620 09:25:52.705632 26428 raft_consensus.cc:388] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:52.705664 26428 raft_consensus.cc:482] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:52.705719 26428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:52.706238 26428 raft_consensus.cc:504] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e39979f3981c4607a05adce3726ca67a" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 34671 } }
I20260620 09:25:52.706321 26428 leader_election.cc:302] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e39979f3981c4607a05adce3726ca67a; no voters:
I20260620 09:25:52.706496 26428 leader_election.cc:288] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:52.706521 26433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:52.706624 26433 raft_consensus.cc:686] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [term 1 LEADER]: Becoming Leader. State: Replica: e39979f3981c4607a05adce3726ca67a, State: Running, Role: LEADER
I20260620 09:25:52.706736 26428 sys_catalog.cc:564] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:52.706753 26433 consensus_queue.cc:237] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e39979f3981c4607a05adce3726ca67a" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 34671 } }
I20260620 09:25:52.707126 26435 sys_catalog.cc:455] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e39979f3981c4607a05adce3726ca67a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e39979f3981c4607a05adce3726ca67a" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 34671 } } }
I20260620 09:25:52.707224 26435 sys_catalog.cc:458] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:52.707123 26433 sys_catalog.cc:455] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e39979f3981c4607a05adce3726ca67a. Latest consensus state: current_term: 1 leader_uuid: "e39979f3981c4607a05adce3726ca67a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e39979f3981c4607a05adce3726ca67a" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 34671 } } }
I20260620 09:25:52.707530 26433 sys_catalog.cc:458] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:52.708205 26440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:52.709055 26440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:52.710505 26432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:52.705013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49021 (local address 127.4.59.62:34671)
0620 09:25:52.705223 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:52.705231 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:52.705255 (+ 24us) server_negotiation.cc:407] Connection header received
0620 09:25:52.705344 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:52.705349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:52.705407 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:52.705540 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:52.705763 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:52.707237 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:52.708411 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:52.708926 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:52.709058 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:52.709765 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:52.709769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:52.709769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:52.709955 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:52.709983 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:52.709994 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:52.710106 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:52.710208 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 09:25:52.710270 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":106,"threads_started":1}
I20260620 09:25:52.710871 26440 catalog_manager.cc:1347] Generated new cluster ID: 5b3125fc682046fa8122f9c0ca529764
I20260620 09:25:52.710934 26440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:52.711412 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:52.722113 26440 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:52.722695 26440 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:52.730715 26440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e39979f3981c4607a05adce3726ca67a: Generated new TSK 0
I20260620 09:25:52.730913 26440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:52.733619 26432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:52.717380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48623 (local address 127.4.59.62:34671)
0620 09:25:52.717416 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:52.717417 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:52.719414 (+ 1997us) server_negotiation.cc:407] Connection header received
0620 09:25:52.719558 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:52.719559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:52.719595 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:52.719625 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:52.720229 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:52.729382 (+ 9153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:52.730445 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:52.730731 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:52.730770 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:52.733346 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:52.733347 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:52.733348 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:52.733352 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:52.733366 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:52.733370 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:52.733398 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:52.733500 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 09:25:52.733542 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":8220,"server-negotiator.queue_time_us":20}
I20260620 09:25:52.735688 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 26360
2026-06-20T09:25:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1307 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:52 dist-test-slave-414w krb5kdc[26459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:52 dist-test-slave-414w krb5kdc[26459](info): set up 2 sockets
Jun 20 09:25:52 dist-test-slave-414w krb5kdc[26459](info): commencing operation
krb5kdc: starting...
W20260620 09:25:53.761556 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:53 dist-test-slave-414w krb5kdc[26459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:25:53.884083 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:37891
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:40253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:37891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:53.963769 26472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:53.964001 26472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:53.964035 26472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:53.965171 26472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:53.965236 26472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:53.965261 26472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:53.965292 26472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:53.965319 26472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:53.966733 26472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:40253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:37891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:37891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:53.967000 26472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:53.967212 26472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:53.971884 26480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:53.971887 26478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:53.971915 26477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:53.972082 26472 server_base.cc:1034] running on GCE node
I20260620 09:25:53.972326 26472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:53.972653 26472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:53.973814 26472 hybrid_clock.cc:648] HybridClock initialized: now 1781947553973807 us; error 37 us; skew 500 ppm
Jun 20 09:25:53 dist-test-slave-414w krb5kdc[26459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947553, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:53.979387 26472 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:25:53.979823 26472 webserver.cc:460] Webserver started at http://127.4.59.62:36595/ using document root <none> and password file <none>
I20260620 09:25:53.980000 26472 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:53.980041 26472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:53.980126 26472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:53.981076 26472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "6aefd0e567a54d629e4bb0cf7467892b"
format_stamp: "Formatted at 2026-06-20 09:25:53 on dist-test-slave-414w"
server_key: "b9c697970d533af234c6fc9dbf97040a"
server_key_iv: "328dec1da6399dc0bec6858c904c35d3"
server_key_version: "encryptionkey@0"
I20260620 09:25:53.981447 26472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "6aefd0e567a54d629e4bb0cf7467892b"
format_stamp: "Formatted at 2026-06-20 09:25:53 on dist-test-slave-414w"
server_key: "b9c697970d533af234c6fc9dbf97040a"
server_key_iv: "328dec1da6399dc0bec6858c904c35d3"
server_key_version: "encryptionkey@0"
I20260620 09:25:53.982658 26472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:53.983311 26487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:53.983479 26472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:53.983564 26472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "6aefd0e567a54d629e4bb0cf7467892b"
format_stamp: "Formatted at 2026-06-20 09:25:53 on dist-test-slave-414w"
server_key: "b9c697970d533af234c6fc9dbf97040a"
server_key_iv: "328dec1da6399dc0bec6858c904c35d3"
server_key_version: "encryptionkey@0"
I20260620 09:25:53.983616 26472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:53.987222 26472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:53.989317 26472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:53.989420 26472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:53.993010 26472 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:37891
I20260620 09:25:53.993039 26539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:37891 every 8 connection(s)
I20260620 09:25:53.993371 26472 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:53.994331 26540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:53.996381 26540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b: Bootstrap starting.
I20260620 09:25:53.996865 26540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:53.997097 26540 log.cc:824] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:53.997782 26540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b: No bootstrap required, opened a new log
I20260620 09:25:53.998394 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 26472
I20260620 09:25:53.998514 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:53.998692 4332 external_mini_cluster.cc:1428] Setting key 93ecbdbd277910d81eecd6b795bd2e20
I20260620 09:25:53.999859 26540 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aefd0e567a54d629e4bb0cf7467892b" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37891 } }
I20260620 09:25:54.000028 26540 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 09:25:54.000066 26540 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aefd0e567a54d629e4bb0cf7467892b, State: Initialized, Role: FOLLOWER
I20260620 09:25:54.000263 26540 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aefd0e567a54d629e4bb0cf7467892b" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37891 } }
I20260620 09:25:54.000380 26540 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 09:25:54.000424 26540 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 09:25:54.000473 26540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:54.001073 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:53.999176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46231 (local address 127.4.59.62:37891)
0620 09:25:53.999355 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:53.999364 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:53.999399 (+ 35us) server_negotiation.cc:407] Connection header received
0620 09:25:53.999821 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:53.999829 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:53.999861 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:54.000006 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:54.000607 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:54.000611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:54.000611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:54.000615 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:54.000659 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:54.000673 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:54.000744 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:54.000800 (+ 56us) server_negotiation.cc:299] Negotiation successful
0620 09:25:54.000844 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
I20260620 09:25:54.001499 26540 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aefd0e567a54d629e4bb0cf7467892b" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37891 } }
I20260620 09:25:54.001614 26540 leader_election.cc:302] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6aefd0e567a54d629e4bb0cf7467892b; no voters:
I20260620 09:25:54.001791 26540 leader_election.cc:288] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 09:25:54.001834 26545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:54.001967 26545 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [term 1 LEADER]: Becoming Leader. State: Replica: 6aefd0e567a54d629e4bb0cf7467892b, State: Running, Role: LEADER
I20260620 09:25:54.002031 26540 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:54.002053 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:54.002060 26545 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aefd0e567a54d629e4bb0cf7467892b" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37891 } }
I20260620 09:25:54.002595 26545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6aefd0e567a54d629e4bb0cf7467892b. Latest consensus state: current_term: 1 leader_uuid: "6aefd0e567a54d629e4bb0cf7467892b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aefd0e567a54d629e4bb0cf7467892b" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37891 } } }
I20260620 09:25:54.002607 26547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6aefd0e567a54d629e4bb0cf7467892b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aefd0e567a54d629e4bb0cf7467892b" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37891 } } }
I20260620 09:25:54.002730 26545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:54.002745 26547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:54.003079 26554 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:54.003621 26554 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:54.005131 26554 catalog_manager.cc:1347] Generated new cluster ID: 2bf1320bc81c455c82291bf72aa7511b
I20260620 09:25:54.005198 26554 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:54.005849 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:54.004574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48337 (local address 127.4.59.62:37891)
0620 09:25:54.004616 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:54.004618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:54.004805 (+ 187us) server_negotiation.cc:407] Connection header received
0620 09:25:54.004899 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:54.004900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:54.004905 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:54.004932 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:54.005633 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:54.005634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:54.005634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 09:25:54.005638 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:54.005651 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 09:25:54.005654 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:54.005709 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:54.005780 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 09:25:54.005791 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 09:25:54.017045 26554 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:54.017205 26554 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:54.021250 26554 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6aefd0e567a54d629e4bb0cf7467892b: Generated new TSK 0
I20260620 09:25:54.021397 26554 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 09:25:54.026513 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 26472
2026-06-20T09:25:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1289 ms)
[----------] 18 tests from AuthTokenIssuingTest (23478 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:54 dist-test-slave-414w krb5kdc[26571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:54 dist-test-slave-414w krb5kdc[26571](info): set up 2 sockets
Jun 20 09:25:54 dist-test-slave-414w krb5kdc[26571](info): commencing operation
krb5kdc: starting...
W20260620 09:25:55.061477 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:55 dist-test-slave-414w krb5kdc[26571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:25:55.179518 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:37593
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:37943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:37593,127.4.59.61:36409,127.4.59.60:36183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:55.258738 26584 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 09:25:55.258970 26584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:55.258998 26584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:55.259042 26584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:55.260123 26584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:55.260186 26584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:55.260210 26584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:55.260237 26584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:55.260267 26584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:55.261706 26584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:37943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:37593,127.4.59.61:36409,127.4.59.60:36183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:37593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:55.261968 26584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:55.262177 26584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:55.266723 26590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:55.266749 26584 server_base.cc:1034] running on GCE node
W20260620 09:25:55.266717 26589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:55.266717 26592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:55.267138 26584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:55.267369 26584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:55.268510 26584 hybrid_clock.cc:648] HybridClock initialized: now 1781947555268501 us; error 29 us; skew 500 ppm
Jun 20 09:25:55 dist-test-slave-414w krb5kdc[26571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947555, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:55.271001 26584 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:25:55.271517 26584 webserver.cc:460] Webserver started at http://127.4.59.62:33621/ using document root <none> and password file <none>
I20260620 09:25:55.271742 26584 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:55.271811 26584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:55.271960 26584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:55.272945 26584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "78f30cb6cc7d43e2a53d8e77f396bf46"
format_stamp: "Formatted at 2026-06-20 09:25:55 on dist-test-slave-414w"
server_key: "da75cd943da1b3560ad2d961d7678909"
server_key_iv: "eaeef28a8e127b0c585fabfc235e3ea8"
server_key_version: "encryptionkey@0"
I20260620 09:25:55.273319 26584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "78f30cb6cc7d43e2a53d8e77f396bf46"
format_stamp: "Formatted at 2026-06-20 09:25:55 on dist-test-slave-414w"
server_key: "da75cd943da1b3560ad2d961d7678909"
server_key_iv: "eaeef28a8e127b0c585fabfc235e3ea8"
server_key_version: "encryptionkey@0"
I20260620 09:25:55.274590 26584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 09:25:55.275327 26599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:55.275508 26584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:25:55.275604 26584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "78f30cb6cc7d43e2a53d8e77f396bf46"
format_stamp: "Formatted at 2026-06-20 09:25:55 on dist-test-slave-414w"
server_key: "da75cd943da1b3560ad2d961d7678909"
server_key_iv: "eaeef28a8e127b0c585fabfc235e3ea8"
server_key_version: "encryptionkey@0"
I20260620 09:25:55.275684 26584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:55.294291 26584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:55.296660 26584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:55.296825 26584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:55.300941 26584 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:37593
I20260620 09:25:55.300978 26651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:37593 every 8 connection(s)
I20260620 09:25:55.301345 26584 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:55.302275 26652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:55.303725 26652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } has no permanent_uuid. Determining permanent_uuid...
I20260620 09:25:55.304090 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 26584
I20260620 09:25:55.304188 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:55.304360 4332 external_mini_cluster.cc:1428] Setting key f05fe7be178b997c20f8f34bfd4da323
Jun 20 09:25:55 dist-test-slave-414w krb5kdc[26571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947555, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.59.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.61@KRBTEST.COM" created.
I20260620 09:25:55.315274 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.304468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:37593 (local address 127.0.0.1:43282)
0620 09:25:55.304653 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.304696 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:55.305464 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:55.305767 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:55.305769 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:55.306242 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:55.306401 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.306405 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.307726 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.307736 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.308937 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.308940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.309514 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.309516 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.309609 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.310143 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:55.310154 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:55.311721 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:55.313788 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.313794 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.313796 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.313987 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.314251 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.314252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.314252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.314337 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.314684 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:55.314689 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:55.314750 (+ 61us) client_negotiation.cc:769] Sending connection context
0620 09:25:55.314860 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 09:25:55.315089 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":91,"thread_start_us":82,"threads_started":1}
I20260620 09:25:55.315274 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.304498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43282 (local address 127.4.59.62:37593)
0620 09:25:55.304660 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.304668 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.304790 (+ 122us) server_negotiation.cc:407] Connection header received
0620 09:25:55.305616 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.305618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.305643 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.305721 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.306554 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.307600 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.309029 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.309394 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.309485 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.311839 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:55.311846 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:55.311847 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:55.311860 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:55.313690 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.314081 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.314091 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.314091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.314172 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.314403 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.314404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.314405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.314612 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:55.314706 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:55.314884 (+ 178us) server_negotiation.cc:299] Negotiation successful
0620 09:25:55.315025 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260620 09:25:55.317112 26652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } has no permanent_uuid. Determining permanent_uuid...
I20260620 09:25:55.317523 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.317392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:36409 (local address 127.0.0.1:44998)
0620 09:25:55.317447 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.317490 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:36409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 09:25:55.317639 26600 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.59.61:36409: connect: Connection refused (error 111)
I20260620 09:25:55.318050 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.317953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:36409 (local address 127.0.0.1:45004)
0620 09:25:55.318005 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.318029 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:36409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 09:25:55.318220 26652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.61:36409: Network error: Client connection negotiation failed: client connection to 127.4.59.61:36409: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.61.keytab.
Entry for principal kudu/127.4.59.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.61.keytab.
I20260620 09:25:55.331456 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.61:36409
--webserver_interface=127.4.59.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.61.keytab
--principal=kudu/127.4.59.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:37943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:37593,127.4.59.61:36409,127.4.59.60:36183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:25:55.367419 26652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } attempt: 1
I20260620 09:25:55.368070 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.367861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:36409 (local address 127.0.0.1:45014)
0620 09:25:55.367993 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.368038 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:36409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260620 09:25:55.368500 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.368404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:36409 (local address 127.0.0.1:45020)
0620 09:25:55.368452 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.368480 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:36409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 09:25:55.368608 26652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.61:36409: Network error: Client connection negotiation failed: client connection to 127.4.59.61:36409: connect: Connection refused (error 111)
W20260620 09:25:55.411929 26659 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 09:25:55.412115 26659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:55.412133 26659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:55.412181 26659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:55.413303 26659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:55.413343 26659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:55.413358 26659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:55.413371 26659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:55.413419 26659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:55.414912 26659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:37943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:37593,127.4.59.61:36409,127.4.59.60:36183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.61.keytab
--principal=kudu/127.4.59.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.61:36409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.59.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:55.415148 26659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:55.415356 26659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:55.419890 26664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:55.419891 26667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:55.419940 26665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:55.420159 26659 server_base.cc:1034] running on GCE node
I20260620 09:25:55.420389 26659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:55.420621 26659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:55.421805 26659 hybrid_clock.cc:648] HybridClock initialized: now 1781947555421797 us; error 36 us; skew 500 ppm
Jun 20 09:25:55 dist-test-slave-414w krb5kdc[26571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947555, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:55.424325 26659 init.cc:377] Logged in from keytab as kudu/127.4.59.61@KRBTEST.COM (short username kudu)
I20260620 09:25:55.424705 26659 webserver.cc:460] Webserver started at http://127.4.59.61:46015/ using document root <none> and password file <none>
I20260620 09:25:55.424877 26659 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:55.424918 26659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:55.425004 26659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:55.426072 26659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/data/instance:
uuid: "6d00e573a1124e1e9ddcabe18378379d"
format_stamp: "Formatted at 2026-06-20 09:25:55 on dist-test-slave-414w"
server_key: "9679ee816beed7411950cdd02ea63a47"
server_key_iv: "eaddf7339a290b149df8b8721dd19302"
server_key_version: "encryptionkey@0"
I20260620 09:25:55.426367 26659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/wal/instance:
uuid: "6d00e573a1124e1e9ddcabe18378379d"
format_stamp: "Formatted at 2026-06-20 09:25:55 on dist-test-slave-414w"
server_key: "9679ee816beed7411950cdd02ea63a47"
server_key_iv: "eaddf7339a290b149df8b8721dd19302"
server_key_version: "encryptionkey@0"
I20260620 09:25:55.427572 26659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:25:55.428263 26674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:55.428447 26659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:55.428526 26659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/wal
uuid: "6d00e573a1124e1e9ddcabe18378379d"
format_stamp: "Formatted at 2026-06-20 09:25:55 on dist-test-slave-414w"
server_key: "9679ee816beed7411950cdd02ea63a47"
server_key_iv: "eaddf7339a290b149df8b8721dd19302"
server_key_version: "encryptionkey@0"
I20260620 09:25:55.428581 26659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:55.436762 26652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } attempt: 2
I20260620 09:25:55.437184 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.437022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:36409 (local address 127.0.0.1:45028)
0620 09:25:55.437082 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.437122 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:36409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 09:25:55.437523 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.437437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:36409 (local address 127.0.0.1:45034)
0620 09:25:55.437482 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.437506 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:36409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 09:25:55.437646 26652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.61:36409: Network error: Client connection negotiation failed: client connection to 127.4.59.61:36409: connect: Connection refused (error 111)
I20260620 09:25:55.443138 26659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:55.445014 26659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:55.445243 26659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:55.448875 26659 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.61:36409
I20260620 09:25:55.448938 26726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.61:36409 every 8 connection(s)
I20260620 09:25:55.449327 26659 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/data/info.pb
I20260620 09:25:55.450284 26727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:55.451761 26727 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } has no permanent_uuid. Determining permanent_uuid...
I20260620 09:25:55.456197 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 26659
I20260620 09:25:55.456266 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-1/wal/instance
I20260620 09:25:55.456440 4332 external_mini_cluster.cc:1428] Setting key bc53c4ab41c4fd6b337ae7fa048c106d
Jun 20 09:25:55 dist-test-slave-414w krb5kdc[26571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947555, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.61@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:55.460777 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.452408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.4.59.62:37593)
0620 09:25:55.452473 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.452475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.452653 (+ 178us) server_negotiation.cc:407] Connection header received
0620 09:25:55.453076 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.453077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.453086 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.453174 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.453726 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.454189 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.455115 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.455298 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.455347 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.457839 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:55.457841 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:55.457841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:55.457850 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:55.459340 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.459838 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.459838 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.459839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.459887 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.460171 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.460171 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.460172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.460265 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:55.460345 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:55.460611 (+ 266us) server_negotiation.cc:299] Negotiation successful
0620 09:25:55.460689 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 09:25:55.460960 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.452383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:37593 (local address 127.0.0.1:43284)
0620 09:25:55.452530 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.452569 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:55.453002 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:55.453255 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:55.453258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:55.453479 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:55.453654 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.453658 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.454271 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.454272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.455045 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.455047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.455377 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.455377 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.455480 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.455945 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:55.455953 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:55.457706 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:55.459426 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.459433 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.459435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.459749 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.459953 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.459954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.459954 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.460105 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.460346 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:55.460354 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:55.460475 (+ 121us) client_negotiation.cc:769] Sending connection context
0620 09:25:55.460578 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 09:25:55.460731 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260620 09:25:55.461493 26727 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.59.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.60@KRBTEST.COM" created.
Jun 20 09:25:55 dist-test-slave-414w krb5kdc[26571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947555, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.61@KRBTEST.COM for kudu/127.4.59.61@KRBTEST.COM
I20260620 09:25:55.470850 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.461740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:36409 (local address 127.0.0.1:45040)
0620 09:25:55.461794 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.461807 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:55.461854 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:55.462064 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:55.462065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:55.462227 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:55.462290 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.462291 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.463092 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.463093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.463928 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.463931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.464397 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.464398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.464476 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.464886 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:55.464890 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:55.468057 (+ 3167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:55.469796 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.469797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.469797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.469986 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.470211 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.470211 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.470212 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.470269 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.470541 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:55.470542 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:55.470578 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 09:25:55.470646 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 09:25:55.470742 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 09:25:55.470942 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.461775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45040 (local address 127.4.59.61:36409)
0620 09:25:55.461918 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.461922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.461941 (+ 19us) server_negotiation.cc:407] Connection header received
0620 09:25:55.461969 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.461970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.461980 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.462039 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.462363 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.463006 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.464030 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.464311 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.464370 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.468156 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:55.468163 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:55.468163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:55.468170 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:55.469721 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.470085 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.470088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.470088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.470144 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.470338 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.470339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.470339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.470469 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:55.470561 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:55.470684 (+ 123us) server_negotiation.cc:299] Negotiation successful
0620 09:25:55.470797 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":116,"thread_start_us":94,"threads_started":1}
I20260620 09:25:55.471441 26727 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } has no permanent_uuid. Determining permanent_uuid...
I20260620 09:25:55.471771 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.471647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:36183 (local address 127.0.0.1:56340)
0620 09:25:55.471692 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.471745 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:36183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 09:25:55.471882 26676 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.59.60:36183: connect: Connection refused (error 111)
I20260620 09:25:55.472301 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.472210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:36183 (local address 127.0.0.1:56356)
0620 09:25:55.472257 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.472280 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:36183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 09:25:55.472404 26727 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.60:36183: Network error: Client connection negotiation failed: client connection to 127.4.59.60:36183: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.60.keytab.
Entry for principal kudu/127.4.59.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.60.keytab.
I20260620 09:25:55.483693 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.60:36183
--webserver_interface=127.4.59.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.60.keytab
--principal=kudu/127.4.59.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:37943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:37593,127.4.59.61:36409,127.4.59.60:36183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:25:55.521622 26727 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } attempt: 1
I20260620 09:25:55.522138 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.522006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:36183 (local address 127.0.0.1:56370)
0620 09:25:55.522071 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.522110 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:36183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 09:25:55.522531 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.522435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:36183 (local address 127.0.0.1:56378)
0620 09:25:55.522485 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.522511 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:36183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 09:25:55.522715 26727 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.60:36183: Network error: Client connection negotiation failed: client connection to 127.4.59.60:36183: connect: Connection refused (error 111)
I20260620 09:25:55.528810 26652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } attempt: 3
Jun 20 09:25:55 dist-test-slave-414w krb5kdc[26571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947555, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for kudu/127.4.59.61@KRBTEST.COM
I20260620 09:25:55.535845 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.529167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45050 (local address 127.4.59.61:36409)
0620 09:25:55.529233 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.529235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.529246 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:25:55.529305 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.529306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.529318 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.529399 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.529705 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.530196 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.530925 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.531140 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.531223 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.533300 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:55.533301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:55.533302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:55.533311 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:55.534719 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.535078 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.535080 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.535080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.535136 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.535334 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.535335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.535335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.535445 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:55.535521 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:55.535677 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 09:25:55.535753 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 09:25:55.535852 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.529101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:36409 (local address 127.0.0.1:45050)
0620 09:25:55.529169 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.529180 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:55.529235 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:55.529408 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:55.529409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:55.529563 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:55.529626 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.529627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.530288 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.530290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.530831 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.530833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.531224 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.531225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.531312 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.531772 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:55.531778 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:55.533199 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:55.534821 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.534822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.534823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.534992 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.535203 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.535203 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.535204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.535263 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.535513 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:55.535514 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:55.535562 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 09:25:55.535653 (+ 91us) client_negotiation.cc:240] Negotiation successful
0620 09:25:55.535761 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 09:25:55.536413 26652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } has no permanent_uuid. Determining permanent_uuid...
I20260620 09:25:55.536748 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.536651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:36183 (local address 127.0.0.1:56380)
0620 09:25:55.536696 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.536724 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:36183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 09:25:55.537068 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.536986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:36183 (local address 127.0.0.1:56390)
0620 09:25:55.537027 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.537050 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:36183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260620 09:25:55.537168 26652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.60:36183: Network error: Client connection negotiation failed: client connection to 127.4.59.60:36183: connect: Connection refused (error 111)
W20260620 09:25:55.565227 26734 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 09:25:55.565435 26734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:55.565454 26734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:55.565475 26734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:55.566558 26734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:55.566593 26734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:55.566607 26734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:55.566620 26734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:55.566632 26734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:55.568069 26734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:37943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:37593,127.4.59.61:36409,127.4.59.60:36183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.60.keytab
--principal=kudu/127.4.59.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.60:36183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.59.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:55.568252 26734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:55.568341 26652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } attempt: 1
I20260620 09:25:55.568487 26734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 09:25:55.568771 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.568625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:36183 (local address 127.0.0.1:56400)
0620 09:25:55.568693 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.568737 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:36183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260620 09:25:55.569255 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.569089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:36183 (local address 127.0.0.1:56406)
0620 09:25:55.569134 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.569217 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:36183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 09:25:55.569375 26652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.60:36183: Network error: Client connection negotiation failed: client connection to 127.4.59.60:36183: connect: Connection refused (error 111)
W20260620 09:25:55.573596 26740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:55.573643 26739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:55.573596 26742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:55.573699 26734 server_base.cc:1034] running on GCE node
I20260620 09:25:55.574045 26734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:55.574326 26734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:55.575505 26734 hybrid_clock.cc:648] HybridClock initialized: now 1781947555575504 us; error 43 us; skew 500 ppm
Jun 20 09:25:55 dist-test-slave-414w krb5kdc[26571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947555, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:55.577981 26734 init.cc:377] Logged in from keytab as kudu/127.4.59.60@KRBTEST.COM (short username kudu)
I20260620 09:25:55.578472 26734 webserver.cc:460] Webserver started at http://127.4.59.60:35701/ using document root <none> and password file <none>
I20260620 09:25:55.578697 26734 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:55.578740 26734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:55.578882 26734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:55.579905 26734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/data/instance:
uuid: "9060766903694cf89918f4216ced1e40"
format_stamp: "Formatted at 2026-06-20 09:25:55 on dist-test-slave-414w"
server_key: "58cf3799287e3a91cb0568cfcdc4e5c5"
server_key_iv: "eb4e6db926de154a03bd26028ca9e3b5"
server_key_version: "encryptionkey@0"
I20260620 09:25:55.580260 26734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/wal/instance:
uuid: "9060766903694cf89918f4216ced1e40"
format_stamp: "Formatted at 2026-06-20 09:25:55 on dist-test-slave-414w"
server_key: "58cf3799287e3a91cb0568cfcdc4e5c5"
server_key_iv: "eb4e6db926de154a03bd26028ca9e3b5"
server_key_version: "encryptionkey@0"
I20260620 09:25:55.581591 26734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 09:25:55.582319 26749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:55.582515 26734 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 09:25:55.582583 26734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/wal
uuid: "9060766903694cf89918f4216ced1e40"
format_stamp: "Formatted at 2026-06-20 09:25:55 on dist-test-slave-414w"
server_key: "58cf3799287e3a91cb0568cfcdc4e5c5"
server_key_iv: "eb4e6db926de154a03bd26028ca9e3b5"
server_key_version: "encryptionkey@0"
I20260620 09:25:55.582669 26734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:55.590854 26727 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } attempt: 2
I20260620 09:25:55.591027 26734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:55.591228 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.591113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:36183 (local address 127.0.0.1:56418)
0620 09:25:55.591175 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.591202 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:36183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 09:25:55.591629 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.591535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:36183 (local address 127.0.0.1:56434)
0620 09:25:55.591581 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.591608 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:36183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 09:25:55.591751 26727 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.60:36183: Network error: Client connection negotiation failed: client connection to 127.4.59.60:36183: connect: Connection refused (error 111)
I20260620 09:25:55.592971 26734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:55.593125 26734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:55.596969 26734 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.60:36183
I20260620 09:25:55.597002 26801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.60:36183 every 8 connection(s)
I20260620 09:25:55.597394 26734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/data/info.pb
I20260620 09:25:55.598035 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 26734
I20260620 09:25:55.598176 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781947262065055-4332-0/minicluster-data/master-2/wal/instance
I20260620 09:25:55.598377 4332 external_mini_cluster.cc:1428] Setting key 72e51db3025410bbe12f42e5e7eecfef
I20260620 09:25:55.598599 26802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:55.600600 26802 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 09:25:55 dist-test-slave-414w krb5kdc[26571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.60@KRBTEST.COM
Jun 20 09:25:55 dist-test-slave-414w krb5kdc[26571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947555, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.60@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:55.608788 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.598848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56438 (local address 127.4.59.60:36183)
0620 09:25:55.599038 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.599046 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.599075 (+ 29us) server_negotiation.cc:407] Connection header received
0620 09:25:55.599679 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.599685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.599728 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.599882 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.600492 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.601632 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.602404 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.602816 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.602949 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.605297 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:55.605304 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:55.605304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:55.605309 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:55.607161 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.607492 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.607494 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.607495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.607626 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.607833 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.607834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.607835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.608139 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:55.608290 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:55.608427 (+ 137us) server_negotiation.cc:299] Negotiation successful
0620 09:25:55.608545 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260620 09:25:55.609074 26806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.601192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:37593 (local address 127.0.0.1:43296)
0620 09:25:55.601328 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.601371 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:55.601612 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:55.601813 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:55.601815 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:55.602071 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:55.602137 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.602139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.602785 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.602786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.603360 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.603362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.603774 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.603775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.603833 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.604272 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:55.604279 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:55.605889 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:55.607658 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.607666 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.607668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.607876 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.608127 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.608128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.608128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.608215 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.608642 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:55.608648 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:55.608700 (+ 52us) client_negotiation.cc:769] Sending connection context
0620 09:25:55.608855 (+ 155us) client_negotiation.cc:240] Negotiation successful
0620 09:25:55.608966 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260620 09:25:55.609400 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.601287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43296 (local address 127.4.59.62:37593)
0620 09:25:55.601336 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.601338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.601588 (+ 250us) server_negotiation.cc:407] Connection header received
0620 09:25:55.601682 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.601683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.601696 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.601773 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.602227 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.602684 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.603458 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.603648 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.603730 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.605969 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:55.605971 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:55.605971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:55.605978 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:55.607558 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.607950 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.607952 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.607952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.608025 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.608350 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.608351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.608352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.608541 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:55.608609 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:55.609070 (+ 461us) server_negotiation.cc:299] Negotiation successful
0620 09:25:55.609309 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 09:25:55.609859 26802 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 09:25:55 dist-test-slave-414w krb5kdc[26571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947555, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.60@KRBTEST.COM for kudu/127.4.59.61@KRBTEST.COM
Jun 20 09:25:55 dist-test-slave-414w krb5kdc[26571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:55.616560 26806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.610108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:36409 (local address 127.0.0.1:45056)
0620 09:25:55.610158 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.610172 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:55.610216 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:55.610347 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:55.610348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:55.610485 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:55.610536 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.610538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.611103 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.611104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.611860 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.611862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.612301 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.612303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.612371 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.612662 (+ 291us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:55.612667 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:55.613943 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:55.615524 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.615525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.615526 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.615689 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.615902 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.615902 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.615903 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.615957 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.616239 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:55.616240 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:55.616288 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 09:25:55.616344 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 09:25:55.616466 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 09:25:55.616570 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.610161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45056 (local address 127.4.59.61:36409)
0620 09:25:55.610200 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.610202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.610211 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:25:55.610288 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.610289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.610296 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.610335 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.610602 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.611027 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.611941 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.612226 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.612277 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.614045 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:55.614046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:55.614047 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:55.614054 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:55.615443 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.615771 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.615772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.615773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.615827 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.616029 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.616030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.616031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.616144 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:55.616223 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:55.616391 (+ 168us) server_negotiation.cc:299] Negotiation successful
0620 09:25:55.616472 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 09:25:55.617151 26802 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } has no permanent_uuid. Determining permanent_uuid...
I20260620 09:25:55.617250 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.609540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43302 (local address 127.4.59.62:37593)
0620 09:25:55.609576 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.609577 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.609586 (+ 9us) server_negotiation.cc:407] Connection header received
0620 09:25:55.609646 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.609647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.609656 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.609716 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.610282 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.610864 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.611805 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.612033 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.612094 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.614691 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:55.614692 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:55.614693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:55.614698 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:55.616125 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.616505 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.616506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.616506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.616557 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.616742 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.616743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.616743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.616853 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:55.617029 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:55.617064 (+ 35us) server_negotiation.cc:299] Negotiation successful
0620 09:25:55.617133 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Jun 20 09:25:55 dist-test-slave-414w krb5kdc[26571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947555, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.60@KRBTEST.COM for kudu/127.4.59.60@KRBTEST.COM
Jun 20 09:25:55 dist-test-slave-414w krb5kdc[26571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.61@KRBTEST.COM
I20260620 09:25:55.623910 26806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.617392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:36183 (local address 127.0.0.1:56454)
0620 09:25:55.617436 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.617446 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:55.617480 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:55.617598 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:55.617599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:55.617770 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:55.617800 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.617801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.618585 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.618586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.619146 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.619148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.619532 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.619533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.619585 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.619905 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:55.619910 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:55.621235 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:55.622813 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.622814 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.622814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.623012 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.623197 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.623198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.623199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.623265 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.623557 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:55.623558 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:55.623602 (+ 44us) client_negotiation.cc:769] Sending connection context
0620 09:25:55.623682 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 09:25:55.623777 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 09:25:55.623914 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.617436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56454 (local address 127.4.59.60:36183)
0620 09:25:55.617469 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.617470 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.617478 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:25:55.617542 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.617542 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.617549 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.617591 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.617892 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.618514 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.619216 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.619464 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.619512 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.621320 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:55.621321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:55.621321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:55.621327 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:55.622736 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.623086 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.623087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.623087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.623136 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.623341 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.623341 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.623342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.623471 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:55.623539 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:55.623710 (+ 171us) server_negotiation.cc:299] Negotiation successful
0620 09:25:55.623821 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":16}
I20260620 09:25:55.625445 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.618092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45060 (local address 127.4.59.61:36409)
0620 09:25:55.618147 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.618149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.618176 (+ 27us) server_negotiation.cc:407] Connection header received
0620 09:25:55.618232 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.618233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.618241 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.618286 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.618798 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.619248 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.620069 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.620254 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.620303 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.622615 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:55.622617 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:55.622617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:55.622624 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:55.624279 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.624654 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.624655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.624656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.624708 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.624915 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.624916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.624917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.625023 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:55.625206 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:55.625250 (+ 44us) server_negotiation.cc:299] Negotiation successful
0620 09:25:55.625321 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 09:25:55.625735 26802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40: Bootstrap starting.
I20260620 09:25:55.626788 26802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:55.626821 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:55.627092 26802 log.cc:824] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:55.627852 26802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40: No bootstrap required, opened a new log
I20260620 09:25:55.627866 26626 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43302
I20260620 09:25:55.629382 26802 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } }
I20260620 09:25:55.629524 26802 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9060766903694cf89918f4216ced1e40, State: Initialized, Role: FOLLOWER
I20260620 09:25:55.629619 26802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } }
I20260620 09:25:55.629910 26808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } } }
I20260620 09:25:55.629976 26802 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:55.630002 26808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 09:25:55.631757 26819 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:25:55.631816 26819 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:25:55.632059 26734 master_runner.cc:186] Error getting master registration for 127.4.59.62:37593: OK, instance_id { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" instance_seqno: 1781947555299728 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 09:25:55.635140 26734 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 09:25:55.644536 26652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } attempt: 2
Jun 20 09:25:55 dist-test-slave-414w krb5kdc[26571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947555, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for kudu/127.4.59.60@KRBTEST.COM
I20260620 09:25:55.653030 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.644911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:36183 (local address 127.0.0.1:56460)
0620 09:25:55.644977 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.644992 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:55.645064 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:55.645299 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:55.645300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:55.645516 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:55.645572 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.645574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.646506 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.646507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.647463 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.647467 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.647953 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.647954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.648009 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.648521 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:55.648526 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:55.650067 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:55.651898 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.651899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.651900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.652119 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.652324 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.652324 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.652325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.652386 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.652713 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:55.652714 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:55.652759 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 09:25:55.652827 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 09:25:55.652922 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260620 09:25:55.653052 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.644997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56460 (local address 127.4.59.60:36183)
0620 09:25:55.645066 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.645069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.645103 (+ 34us) server_negotiation.cc:407] Connection header received
0620 09:25:55.645131 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.645132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.645200 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.645255 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.645663 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.646419 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.647562 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.647869 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.647944 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.650165 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:55.650167 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:55.650167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:55.650175 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:55.651813 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.652193 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.652194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.652195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.652255 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.652453 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.652454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.652455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.652642 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:55.652716 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:55.652867 (+ 151us) server_negotiation.cc:299] Negotiation successful
0620 09:25:55.652958 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 09:25:55.654742 26652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46: Bootstrap starting.
I20260620 09:25:55.655504 26652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:55.655815 26652 log.cc:824] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:55.656688 26652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46: No bootstrap required, opened a new log
I20260620 09:25:55.658133 26652 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } }
I20260620 09:25:55.658289 26652 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78f30cb6cc7d43e2a53d8e77f396bf46, State: Initialized, Role: FOLLOWER
I20260620 09:25:55.658371 26652 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } }
I20260620 09:25:55.658689 26652 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:55.658658 26822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } } }
I20260620 09:25:55.658759 26822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:25:55.658849 26626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 09:25:55.658908 26626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:55.659674 26626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } }
I20260620 09:25:55.660429 26626 leader_election.cc:288] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [CANDIDATE]: Term 1 election: Requested vote from peers 6d00e573a1124e1e9ddcabe18378379d (127.4.59.61:36409), 9060766903694cf89918f4216ced1e40 (127.4.59.60:36183)
W20260620 09:25:55.660596 26833 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:25:55.660704 26833 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:25:55.660665 26776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9060766903694cf89918f4216ced1e40"
I20260620 09:25:55.660784 26776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:55.662086 26776 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78f30cb6cc7d43e2a53d8e77f396bf46 in term 1.
I20260620 09:25:55.663087 26601 leader_election.cc:302] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78f30cb6cc7d43e2a53d8e77f396bf46, 9060766903694cf89918f4216ced1e40; no voters:
I20260620 09:25:55.663251 26822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:55.663383 26822 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [term 1 LEADER]: Becoming Leader. State: Replica: 78f30cb6cc7d43e2a53d8e77f396bf46, State: Running, Role: LEADER
I20260620 09:25:55.663470 26822 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } }
I20260620 09:25:55.663885 26822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78f30cb6cc7d43e2a53d8e77f396bf46. Latest consensus state: current_term: 1 leader_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } } }
I20260620 09:25:55.663966 26822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:55.663662 26584 master_runner.cc:186] Error getting master registration for 127.4.59.61:36409: OK, instance_id { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" instance_seqno: 1781947555447890 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 09:25:55.666208 26584 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 09:25:55.674341 26841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:55.674602 26841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:55.676196 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.666488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43306 (local address 127.4.59.62:37593)
0620 09:25:55.666531 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.666533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.667139 (+ 606us) server_negotiation.cc:407] Connection header received
0620 09:25:55.667244 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.667245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.667257 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.667289 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.668060 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.668878 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.670068 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.670392 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.670436 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.672330 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:55.672331 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:55.672332 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:55.672339 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:55.673958 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.675373 (+ 1415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.675374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.675375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.675434 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.675638 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.675639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.675639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.675760 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:55.675833 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:55.676032 (+ 199us) server_negotiation.cc:299] Negotiation successful
0620 09:25:55.676106 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 09:25:55.677852 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.664252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45062 (local address 127.4.59.61:36409)
0620 09:25:55.664296 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.664298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.666748 (+ 2450us) server_negotiation.cc:407] Connection header received
0620 09:25:55.666897 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.666898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.666910 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.666954 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.668169 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.668898 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.670529 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.670844 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.670893 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.673661 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:55.673662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:55.673663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:55.673680 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:55.675505 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.676065 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.676066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.676067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.676132 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.676659 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.676660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.676660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.676802 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:55.677382 (+ 580us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:55.677681 (+ 299us) server_negotiation.cc:299] Negotiation successful
0620 09:25:55.677757 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 09:25:55.682209 26776 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40 [term 1 FOLLOWER]: Refusing update from remote peer 78f30cb6cc7d43e2a53d8e77f396bf46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:25:55.682521 26822 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:25:55.682888 26727 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } attempt: 3
I20260620 09:25:55.683784 26808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78f30cb6cc7d43e2a53d8e77f396bf46. Latest consensus state: current_term: 1 leader_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } } }
I20260620 09:25:55.683871 26808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:25:55.685580 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.666819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56470 (local address 127.4.59.60:36183)
0620 09:25:55.666858 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.666860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.676846 (+ 9986us) server_negotiation.cc:407] Connection header received
0620 09:25:55.676931 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.676932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.676943 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.676972 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.677558 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.678304 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.679625 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.679933 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.680019 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.681461 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:55.681463 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:55.681463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:55.681470 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:55.684303 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.684689 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.684691 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.684691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.684747 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.684979 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.684979 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.684980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.685099 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:55.685211 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:55.685417 (+ 206us) server_negotiation.cc:299] Negotiation successful
0620 09:25:55.685493 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Jun 20 09:25:55 dist-test-slave-414w krb5kdc[26571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947555, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.61@KRBTEST.COM for kudu/127.4.59.60@KRBTEST.COM
I20260620 09:25:55.690668 26839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } } }
I20260620 09:25:55.690800 26839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:55.691002 26841 catalog_manager.cc:1347] Generated new cluster ID: c3e695705610439489a7cf2c697e6324
I20260620 09:25:55.691051 26841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:55.691246 26808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } } }
I20260620 09:25:55.691325 26808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:25:55.691680 26847 mvcc.cc:205] Tried to move back new op lower bound from 7298857188064980992 to 7298857187998474240. Current Snapshot: MvccSnapshot[applied={T|T < 7298857188064980992}]
I20260620 09:25:55.692035 26845 mvcc.cc:205] Tried to move back new op lower bound from 7298857188064980992 to 7298857187998474240. Current Snapshot: MvccSnapshot[applied={T|T < 7298857188064980992 or (T in {7298857188064980992})}]
I20260620 09:25:55.700628 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.683110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:36183 (local address 127.0.0.1:56484)
0620 09:25:55.683184 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:55.683194 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:55.683228 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:55.684692 (+ 1464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:55.684693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:55.684841 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:55.684883 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.684884 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.686872 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.686874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.687695 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:55.687697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.688511 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.688512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:55.688570 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.689079 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:55.689084 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:55.690746 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:55.699503 (+ 8757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.699504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.699505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.699725 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.699955 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:55.699956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:55.699956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:55.700018 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:55.700289 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:55.700290 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:55.700343 (+ 53us) client_negotiation.cc:769] Sending connection context
0620 09:25:55.700420 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 09:25:55.700530 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260620 09:25:55.700647 26846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:55.683373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56484 (local address 127.4.59.60:36183)
0620 09:25:55.684483 (+ 1110us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.684485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.684492 (+ 7us) server_negotiation.cc:407] Connection header received
0620 09:25:55.684508 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.684509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.684563 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.684970 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.685035 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.686752 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.687843 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:55.688388 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:55.689660 (+ 1272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:55.697204 (+ 7544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:55.697205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:55.697205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:55.697213 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:55.699429 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.699797 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.699798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.699799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.699880 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:55.700076 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:55.700077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:55.700077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:55.700214 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:55.700286 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:55.700456 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 09:25:55.700530 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1059,"thread_start_us":110,"threads_started":1}
I20260620 09:25:55.702203 26727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d: Bootstrap starting.
I20260620 09:25:55.702960 26727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:55.703261 26727 log.cc:824] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:55.705937 26727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d: No bootstrap required, opened a new log
I20260620 09:25:55.707947 26727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } }
I20260620 09:25:55.708087 26727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d00e573a1124e1e9ddcabe18378379d, State: Initialized, Role: FOLLOWER
I20260620 09:25:55.708177 26727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } }
I20260620 09:25:55.708498 26727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:55.709216 26701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6d00e573a1124e1e9ddcabe18378379d"
I20260620 09:25:55.709323 26701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:55.710330 26701 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78f30cb6cc7d43e2a53d8e77f396bf46 in term 1.
I20260620 09:25:55.711643 26700 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d [term 1 FOLLOWER]: Refusing update from remote peer 78f30cb6cc7d43e2a53d8e77f396bf46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:25:55.712101 26822 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:25:55.713718 26841 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:55.713939 26852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } } }
I20260620 09:25:55.714107 26852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:25:55.714304 26852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78f30cb6cc7d43e2a53d8e77f396bf46. Latest consensus state: current_term: 1 leader_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } } }
I20260620 09:25:55.714403 26852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:25:55.714460 26841 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:55.714484 26852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } } }
I20260620 09:25:55.714614 26852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d [sys.catalog]: This master's current role is: FOLLOWER
W20260620 09:25:55.718117 26867 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:25:55.718219 26867 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:25:55.721917 26839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f30cb6cc7d43e2a53d8e77f396bf46" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37593 } } peers { permanent_uuid: "6d00e573a1124e1e9ddcabe18378379d" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 36409 } } peers { permanent_uuid: "9060766903694cf89918f4216ced1e40" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 36183 } } }
I20260620 09:25:55.722008 26839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:55.722075 26860 mvcc.cc:205] Tried to move back new op lower bound from 7298857188064980992 to 7298857187998474240. Current Snapshot: MvccSnapshot[applied={T|T < 7298857188064980992}]
I20260620 09:25:55.725287 26841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78f30cb6cc7d43e2a53d8e77f396bf46: Generated new TSK 0
I20260620 09:25:55.725500 26841 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 09:25:55.751520 26655 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:55.749811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43314 (local address 127.4.59.62:37593)
0620 09:25:55.749871 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.749874 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.750847 (+ 973us) server_negotiation.cc:407] Connection header received
0620 09:25:55.750960 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.750962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.750974 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.751007 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.751453 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43314: BlockingRecv error: recv got EOF from 127.0.0.1:43314 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 09:25:55.752475 26846 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:55.750201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56488 (local address 127.4.59.60:36183)
0620 09:25:55.750260 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.750262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.751950 (+ 1688us) server_negotiation.cc:407] Connection header received
0620 09:25:55.752052 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.752053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.752063 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.752094 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.752413 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56488: BlockingRecv error: recv got EOF from 127.0.0.1:56488 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 09:25:55.753366 26731 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:55.750576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45078 (local address 127.4.59.61:36409)
0620 09:25:55.750617 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.750619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.752813 (+ 2194us) server_negotiation.cc:407] Connection header received
0620 09:25:55.752918 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.752919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.752931 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.752963 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.753304 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45078: BlockingRecv error: recv got EOF from 127.0.0.1:45078 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 09:25:55.755846 26731 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:55.755076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45092 (local address 127.4.59.61:36409)
0620 09:25:55.755112 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.755114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.755333 (+ 219us) server_negotiation.cc:407] Connection header received
0620 09:25:55.755444 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.755445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.755462 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.755493 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.755800 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45092: BlockingRecv error: recv got EOF from 127.0.0.1:45092 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 09:25:55.758924 26731 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:55.758146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45094 (local address 127.4.59.61:36409)
0620 09:25:55.758179 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.758181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.758413 (+ 232us) server_negotiation.cc:407] Connection header received
0620 09:25:55.758515 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.758516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.758526 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.758557 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.758877 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45094: BlockingRecv error: recv got EOF from 127.0.0.1:45094 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260620 09:25:55.762867 26731 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:55.762109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45100 (local address 127.4.59.61:36409)
0620 09:25:55.762156 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.762158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.762389 (+ 231us) server_negotiation.cc:407] Connection header received
0620 09:25:55.762489 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.762490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.762499 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.762529 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.762827 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45100: BlockingRecv error: recv got EOF from 127.0.0.1:45100 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260620 09:25:55.768795 26731 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:55.768038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45112 (local address 127.4.59.61:36409)
0620 09:25:55.768074 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.768075 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.768296 (+ 221us) server_negotiation.cc:407] Connection header received
0620 09:25:55.768395 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.768396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.768405 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.768436 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.768754 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45112: BlockingRecv error: recv got EOF from 127.0.0.1:45112 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260620 09:25:55.779461 26731 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:55.778469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45128 (local address 127.4.59.61:36409)
0620 09:25:55.778509 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.778511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.778807 (+ 296us) server_negotiation.cc:407] Connection header received
0620 09:25:55.778918 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.778919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.778930 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.778963 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.779395 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45128: BlockingRecv error: recv got EOF from 127.0.0.1:45128 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260620 09:25:55.805024 26731 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:55.804082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45140 (local address 127.4.59.61:36409)
0620 09:25:55.804129 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.804131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.804405 (+ 274us) server_negotiation.cc:407] Connection header received
0620 09:25:55.804525 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.804527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.804540 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.804574 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.804964 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45140: BlockingRecv error: recv got EOF from 127.0.0.1:45140 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260620 09:25:55.839377 26731 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:55.838547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45154 (local address 127.4.59.61:36409)
0620 09:25:55.838598 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.838600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.838742 (+ 142us) server_negotiation.cc:407] Connection header received
0620 09:25:55.838807 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.838808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.838820 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.838900 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.839298 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45154: BlockingRecv error: recv got EOF from 127.0.0.1:45154 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260620 09:25:55.905368 26731 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:55.904539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45156 (local address 127.4.59.61:36409)
0620 09:25:55.904626 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:55.904628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:55.904751 (+ 123us) server_negotiation.cc:407] Connection header received
0620 09:25:55.904832 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:55.904833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:55.904845 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:55.904922 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:55.905305 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45156: BlockingRecv error: recv got EOF from 127.0.0.1:45156 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260620 09:25:56.035275 26731 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:56.034528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45170 (local address 127.4.59.61:36409)
0620 09:25:56.034630 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:56.034632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:56.034682 (+ 50us) server_negotiation.cc:407] Connection header received
0620 09:25:56.034750 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:56.034751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:56.034762 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:56.034805 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:56.035226 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45170: BlockingRecv error: recv got EOF from 127.0.0.1:45170 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260620 09:25:56.293097 26731 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:56.292416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45182 (local address 127.4.59.61:36409)
0620 09:25:56.292483 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:56.292485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:56.292533 (+ 48us) server_negotiation.cc:407] Connection header received
0620 09:25:56.292584 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:56.292584 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:56.292594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:56.292651 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:56.293032 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45182: BlockingRecv error: recv got EOF from 127.0.0.1:45182 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 09:25:56.632344 26819 catalog_manager.cc:1259] Loaded cluster ID: c3e695705610439489a7cf2c697e6324
I20260620 09:25:56.632416 26819 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40: loading cluster ID for follower catalog manager: success
I20260620 09:25:56.633275 26819 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40: acquiring CA information for follower catalog manager: success
I20260620 09:25:56.633582 26819 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9060766903694cf89918f4216ced1e40: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 09:25:56.718734 26867 catalog_manager.cc:1259] Loaded cluster ID: c3e695705610439489a7cf2c697e6324
I20260620 09:25:56.718824 26867 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d: loading cluster ID for follower catalog manager: success
I20260620 09:25:56.719666 26867 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d: acquiring CA information for follower catalog manager: success
I20260620 09:25:56.719978 26867 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6d00e573a1124e1e9ddcabe18378379d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 09:25:56.809401 26935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:56.806289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45198 (local address 127.4.59.61:36409)
0620 09:25:56.806421 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:56.806424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:56.806493 (+ 69us) server_negotiation.cc:407] Connection header received
0620 09:25:56.806595 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:56.806596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:56.806664 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:56.806760 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:25:56.806912 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:56.807422 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:56.808346 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:56.808603 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:56.808668 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:56.808964 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:25:56.809078 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:25:56.809230 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:56.809270 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 09:25:56.809299 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":55,"threads_started":1}
I20260620 09:25:56.813720 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:56.811236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56496 (local address 127.4.59.60:36183)
0620 09:25:56.811352 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:56.811354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:56.811362 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:25:56.811396 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:56.811396 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:56.811429 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:56.811509 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:25:56.811730 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:56.812205 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:56.812963 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:56.813223 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:56.813284 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:56.813407 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:25:56.813495 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:25:56.813543 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:56.813583 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 09:25:56.813600 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":49,"threads_started":1}
I20260620 09:25:56.818351 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:56.815501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56506 (local address 127.4.59.60:36183)
0620 09:25:56.815565 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:56.815567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:56.815574 (+ 7us) server_negotiation.cc:407] Connection header received
0620 09:25:56.815632 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:56.815632 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:56.815643 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:56.815688 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:25:56.815894 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:56.816594 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:56.817526 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:56.817782 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:56.817840 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:56.818027 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:25:56.818061 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:25:56.818128 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:56.818229 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 09:25:56.818251 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 09:25:56.818622 26948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:56.815345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43320 (local address 127.4.59.62:37593)
0620 09:25:56.815562 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:56.815564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:56.815664 (+ 100us) server_negotiation.cc:407] Connection header received
0620 09:25:56.815745 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:56.815746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:56.815805 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:56.815882 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:25:56.816050 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:56.816667 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:56.817690 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:56.818018 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:56.818083 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:56.818243 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:25:56.818360 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:25:56.818418 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:56.818522 (+ 104us) server_negotiation.cc:299] Negotiation successful
0620 09:25:56.818543 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":49,"threads_started":1}
I20260620 09:25:56.818645 26935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:56.815421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45200 (local address 127.4.59.61:36409)
0620 09:25:56.815460 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:56.815462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:56.815530 (+ 68us) server_negotiation.cc:407] Connection header received
0620 09:25:56.815620 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:56.815621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:56.815635 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:56.815662 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:25:56.815806 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:56.816554 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:56.817689 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:56.818067 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:56.818111 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:56.818320 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:25:56.818351 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:25:56.818411 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:56.818561 (+ 150us) server_negotiation.cc:299] Negotiation successful
0620 09:25:56.818583 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:25:56.821201 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 26584
I20260620 09:25:56.826669 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 26659
I20260620 09:25:56.832095 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 26734
2026-06-20T09:25:56Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2807 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 09:25:56 dist-test-slave-414w krb5kdc[26952](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 09:25:56 dist-test-slave-414w krb5kdc[26952](info): set up 2 sockets
Jun 20 09:25:56 dist-test-slave-414w krb5kdc[26952](info): commencing operation
krb5kdc: starting...
W20260620 09:25:57.877624 4332 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/test-user.keytab.
Jun 20 09:25:57 dist-test-slave-414w krb5kdc[26952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T09:25:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T09:25:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.59.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
Entry for principal kudu/127.4.59.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab.
I20260620 09:25:57.993450 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:37875
--webserver_interface=127.4.59.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:37875,127.4.59.61:32903,127.4.59.60:39665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 09:25:58.072427 26965 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 09:25:58.072616 26965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:58.072634 26965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:58.072654 26965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:58.073709 26965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:58.073742 26965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:58.073756 26965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:58.073769 26965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:58.073783 26965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:58.075126 26965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:37875,127.4.59.61:32903,127.4.59.60:39665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.62.keytab
--principal=kudu/127.4.59.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.62:37875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.59.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:58.075305 26965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:58.075459 26965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:58.079929 26971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:58.080070 26965 server_base.cc:1034] running on GCE node
W20260620 09:25:58.079928 26970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:58.080154 26973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:58.080442 26965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:58.080708 26965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:58.081840 26965 hybrid_clock.cc:648] HybridClock initialized: now 1781947558081829 us; error 30 us; skew 500 ppm
Jun 20 09:25:58 dist-test-slave-414w krb5kdc[26952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947558, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:58.084218 26965 init.cc:377] Logged in from keytab as kudu/127.4.59.62@KRBTEST.COM (short username kudu)
I20260620 09:25:58.084609 26965 webserver.cc:460] Webserver started at http://127.4.59.62:44767/ using document root <none> and password file <none>
I20260620 09:25:58.084826 26965 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:58.084872 26965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:58.084991 26965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:58.086014 26965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/data/instance:
uuid: "80f6ed82ef1048b88bb83d631ae93fc4"
format_stamp: "Formatted at 2026-06-20 09:25:58 on dist-test-slave-414w"
server_key: "344f8a120d8d0fec5f89e3e3eda70734"
server_key_iv: "54375b0655596cfef723f78b73f7ee16"
server_key_version: "encryptionkey@0"
I20260620 09:25:58.086354 26965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/wal/instance:
uuid: "80f6ed82ef1048b88bb83d631ae93fc4"
format_stamp: "Formatted at 2026-06-20 09:25:58 on dist-test-slave-414w"
server_key: "344f8a120d8d0fec5f89e3e3eda70734"
server_key_iv: "54375b0655596cfef723f78b73f7ee16"
server_key_version: "encryptionkey@0"
I20260620 09:25:58.087690 26965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 09:25:58.088371 26980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:58.088529 26965 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:58.088629 26965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/wal
uuid: "80f6ed82ef1048b88bb83d631ae93fc4"
format_stamp: "Formatted at 2026-06-20 09:25:58 on dist-test-slave-414w"
server_key: "344f8a120d8d0fec5f89e3e3eda70734"
server_key_iv: "54375b0655596cfef723f78b73f7ee16"
server_key_version: "encryptionkey@0"
I20260620 09:25:58.088699 26965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:58.101626 26965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:58.103470 26965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:58.103622 26965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:58.107291 26965 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.62:37875
I20260620 09:25:58.107335 27032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.62:37875 every 8 connection(s)
I20260620 09:25:58.107705 26965 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/data/info.pb
I20260620 09:25:58.108657 27033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:58.110167 27033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 09:25:58 dist-test-slave-414w krb5kdc[26952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947558, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:58.117542 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 26965
I20260620 09:25:58.117647 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-0/wal/instance
I20260620 09:25:58.117861 4332 external_mini_cluster.cc:1428] Setting key 1e65a03827a725c675a3c9c9c78d2d1e
I20260620 09:25:58.119230 27036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.110835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46618 (local address 127.4.59.62:37875)
0620 09:25:58.110988 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.110995 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.111081 (+ 86us) server_negotiation.cc:407] Connection header received
0620 09:25:58.111545 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.111546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.111562 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.111672 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.112135 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.112920 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.113756 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.114056 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.114133 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.116140 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:58.116146 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:58.116146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:58.116152 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:58.117568 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.118051 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.118053 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.118053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.118127 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.118376 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.118377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.118377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.118562 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:58.118623 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:58.118875 (+ 252us) server_negotiation.cc:299] Negotiation successful
0620 09:25:58.119044 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260620 09:25:58.119220 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.110769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:37875 (local address 127.0.0.1:46618)
0620 09:25:58.110945 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.110986 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:58.111425 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:58.111681 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:58.111683 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:58.111909 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:58.112053 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.112057 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.112999 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.112999 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.113684 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.113686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.114151 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.114152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.114207 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.114714 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:58.114723 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:58.116060 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:58.117673 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.117679 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.117681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.117970 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.118200 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.118201 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.118201 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.118309 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.118628 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:58.118634 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:58.118708 (+ 74us) client_negotiation.cc:769] Sending connection context
0620 09:25:58.118813 (+ 105us) client_negotiation.cc:240] Negotiation successful
0620 09:25:58.118988 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260620 09:25:58.120323 27033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } has no permanent_uuid. Determining permanent_uuid...
I20260620 09:25:58.120606 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.120525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:32903 (local address 127.0.0.1:50072)
0620 09:25:58.120554 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.120590 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:32903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260620 09:25:58.120725 26984 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.59.61:32903: connect: Connection refused (error 111)
I20260620 09:25:58.121187 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.121057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:32903 (local address 127.0.0.1:50082)
0620 09:25:58.121100 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.121123 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:32903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 09:25:58.121289 27033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.61:32903: Network error: Client connection negotiation failed: client connection to 127.4.59.61:32903: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.4.59.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.61.keytab.
Entry for principal kudu/127.4.59.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.61.keytab.
I20260620 09:25:58.141047 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.61:32903
--webserver_interface=127.4.59.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.61.keytab
--principal=kudu/127.4.59.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:37875,127.4.59.61:32903,127.4.59.60:39665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:25:58.170517 27033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } attempt: 1
I20260620 09:25:58.171016 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.170879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:32903 (local address 127.0.0.1:50092)
0620 09:25:58.170944 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.170985 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:32903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260620 09:25:58.171347 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.171254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:32903 (local address 127.0.0.1:50108)
0620 09:25:58.171298 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.171322 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:32903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 09:25:58.171425 27033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.61:32903: Network error: Client connection negotiation failed: client connection to 127.4.59.61:32903: connect: Connection refused (error 111)
W20260620 09:25:58.220054 27040 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 09:25:58.220278 27040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:58.220306 27040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:58.220348 27040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:58.221590 27040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:58.221652 27040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:58.221676 27040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:58.221704 27040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:58.221732 27040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:58.223151 27040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:37875,127.4.59.61:32903,127.4.59.60:39665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.61.keytab
--principal=kudu/127.4.59.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.61:32903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.59.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:58.223413 27040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:58.223623 27040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:58.228242 27045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:58.228252 27046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:58.228259 27048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:58.228506 27040 server_base.cc:1034] running on GCE node
I20260620 09:25:58.228794 27040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:58.229072 27040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:58.230235 27040 hybrid_clock.cc:648] HybridClock initialized: now 1781947558230225 us; error 30 us; skew 500 ppm
Jun 20 09:25:58 dist-test-slave-414w krb5kdc[26952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947558, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:58.232671 27040 init.cc:377] Logged in from keytab as kudu/127.4.59.61@KRBTEST.COM (short username kudu)
I20260620 09:25:58.233085 27040 webserver.cc:460] Webserver started at http://127.4.59.61:34699/ using document root <none> and password file <none>
I20260620 09:25:58.233330 27040 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:58.233399 27040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:58.233547 27040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:58.234541 27040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/data/instance:
uuid: "0c5a552c9add4fcf9ed4e681e8aace72"
format_stamp: "Formatted at 2026-06-20 09:25:58 on dist-test-slave-414w"
server_key: "0738cec42f69447cafa73e75c10d0e2a"
server_key_iv: "570bbd98b11b558a210075bcfba5f99e"
server_key_version: "encryptionkey@0"
I20260620 09:25:58.234910 27040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/wal/instance:
uuid: "0c5a552c9add4fcf9ed4e681e8aace72"
format_stamp: "Formatted at 2026-06-20 09:25:58 on dist-test-slave-414w"
server_key: "0738cec42f69447cafa73e75c10d0e2a"
server_key_iv: "570bbd98b11b558a210075bcfba5f99e"
server_key_version: "encryptionkey@0"
I20260620 09:25:58.236186 27040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:25:58.236924 27055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:58.237123 27040 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 09:25:58.237267 27040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/wal
uuid: "0c5a552c9add4fcf9ed4e681e8aace72"
format_stamp: "Formatted at 2026-06-20 09:25:58 on dist-test-slave-414w"
server_key: "0738cec42f69447cafa73e75c10d0e2a"
server_key_iv: "570bbd98b11b558a210075bcfba5f99e"
server_key_version: "encryptionkey@0"
I20260620 09:25:58.237355 27040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:58.239559 27033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } attempt: 2
I20260620 09:25:58.239930 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.239813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:32903 (local address 127.0.0.1:50112)
0620 09:25:58.239869 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.239905 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:32903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 09:25:58.240249 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.240160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:32903 (local address 127.0.0.1:50126)
0620 09:25:58.240202 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.240225 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.61:32903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260620 09:25:58.240330 27033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.61:32903: Network error: Client connection negotiation failed: client connection to 127.4.59.61:32903: connect: Connection refused (error 111)
I20260620 09:25:58.249284 27040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:58.251511 27040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:58.251695 27040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:58.255829 27040 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.61:32903
I20260620 09:25:58.255858 27107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.61:32903 every 8 connection(s)
I20260620 09:25:58.256184 27040 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/data/info.pb
I20260620 09:25:58.257218 27108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:58.258762 27108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 09:25:58 dist-test-slave-414w krb5kdc[26952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947558, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.61@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:58.265324 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 27040
I20260620 09:25:58.265429 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-1/wal/instance
I20260620 09:25:58.265617 4332 external_mini_cluster.cc:1428] Setting key 2d12e4ee05436e56858d145feb272400
I20260620 09:25:58.267863 27036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.259461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46634 (local address 127.4.59.62:37875)
0620 09:25:58.259503 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.259505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.259737 (+ 232us) server_negotiation.cc:407] Connection header received
0620 09:25:58.260240 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.260242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.260256 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.260311 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.260899 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.261447 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.262433 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.262637 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.262722 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.264803 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:58.264804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:58.264804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:58.264811 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:58.266408 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.266896 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.266897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.266898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.266968 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.267213 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.267213 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.267214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.267360 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:58.267432 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:58.267656 (+ 224us) server_negotiation.cc:299] Negotiation successful
0620 09:25:58.267742 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 09:25:58.267895 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.259423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:37875 (local address 127.0.0.1:46634)
0620 09:25:58.259618 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.259651 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:58.260134 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:58.260385 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:58.260388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:58.260626 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:58.260806 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.260809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.261540 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.261541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.262331 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.262333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.262748 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.262749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.262872 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.263382 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:58.263390 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:58.264695 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:58.266490 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.266497 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.266499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.266811 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.267053 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.267053 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.267054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.267148 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.267425 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:58.267429 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:58.267536 (+ 107us) client_negotiation.cc:769] Sending connection context
0620 09:25:58.267615 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 09:25:58.267731 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":87,"threads_started":1}
I20260620 09:25:58.268528 27108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 09:25:58 dist-test-slave-414w krb5kdc[26952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947558, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.61@KRBTEST.COM for kudu/127.4.59.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.59.60@KRBTEST.COM; defaulting to no policy
I20260620 09:25:58.275401 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.268817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:32903 (local address 127.0.0.1:50142)
0620 09:25:58.269018 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.269029 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:58.269091 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:58.269280 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:58.269280 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:58.269418 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:58.269458 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.269460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.270285 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.270286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.270780 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.270781 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.271258 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.271259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.271320 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.271718 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:58.271723 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:58.272815 (+ 1092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:58.274474 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.274475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.274475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.274636 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.274816 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.274817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.274817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.274870 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.275120 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:58.275121 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:58.275168 (+ 47us) client_negotiation.cc:769] Sending connection context
0620 09:25:58.275255 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 09:25:58.275341 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260620 09:25:58.275496 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.268883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50142 (local address 127.4.59.61:32903)
0620 09:25:58.269029 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.269034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.269076 (+ 42us) server_negotiation.cc:407] Connection header received
0620 09:25:58.269163 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.269166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.269180 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.269278 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.269533 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.270195 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.270871 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.271171 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.271258 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.272895 (+ 1637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:58.272902 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:58.272902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:58.272908 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:58.274407 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.274704 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.274706 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.274706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.274753 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.274924 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.274925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.274925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.275041 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:58.275098 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:58.275284 (+ 186us) server_negotiation.cc:299] Negotiation successful
0620 09:25:58.275386 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.59.60@KRBTEST.COM" created.
I20260620 09:25:58.276093 27108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } has no permanent_uuid. Determining permanent_uuid...
I20260620 09:25:58.276412 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.276300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39665 (local address 127.0.0.1:47564)
0620 09:25:58.276345 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.276386 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 09:25:58.276647 27057 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39665: connect: Connection refused (error 111)
I20260620 09:25:58.277027 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.276942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39665 (local address 127.0.0.1:47574)
0620 09:25:58.276987 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.277011 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 09:25:58.277168 27108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.60:39665: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39665: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.59.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.60.keytab.
Entry for principal kudu/127.4.59.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.60.keytab.
I20260620 09:25:58.291915 4332 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Fx1M2/build/release/bin/kudu
/tmp/dist-test-task0Fx1M2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.60:39665
--webserver_interface=127.4.59.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.60.keytab
--principal=kudu/127.4.59.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.59.20:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.59.62:37875,127.4.59.61:32903,127.4.59.60:39665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 09:25:58.326437 27108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } attempt: 1
I20260620 09:25:58.327024 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.326851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39665 (local address 127.0.0.1:47586)
0620 09:25:58.326925 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.326977 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 09:25:58.327456 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.327336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39665 (local address 127.0.0.1:47598)
0620 09:25:58.327401 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.327431 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260620 09:25:58.327603 27108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.60:39665: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39665: connect: Connection refused (error 111)
I20260620 09:25:58.331470 27033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } attempt: 3
Jun 20 09:25:58 dist-test-slave-414w krb5kdc[26952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947558, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for kudu/127.4.59.61@KRBTEST.COM
I20260620 09:25:58.338546 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.331883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50146 (local address 127.4.59.61:32903)
0620 09:25:58.332010 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.332012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.332018 (+ 6us) server_negotiation.cc:407] Connection header received
0620 09:25:58.332039 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.332040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.332047 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.332110 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.332428 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.332930 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.333713 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.333923 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.334010 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.336023 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:58.336025 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:58.336025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:58.336032 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:58.337426 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.337793 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.337794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.337795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.337852 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.338046 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.338047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.338047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.338159 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:58.338235 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:58.338384 (+ 149us) server_negotiation.cc:299] Negotiation successful
0620 09:25:58.338455 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260620 09:25:58.338549 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.331827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:32903 (local address 127.0.0.1:50146)
0620 09:25:58.331882 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.331894 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:58.331951 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:58.332123 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:58.332124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:58.332284 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:58.332347 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.332348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.333020 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.333021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.333610 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.333612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.334009 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.334010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.334094 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.334571 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:58.334575 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:58.335935 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:58.337537 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.337538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.337538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.337707 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.337919 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.337920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.337921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.337981 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.338228 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:58.338228 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:58.338273 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 09:25:58.338360 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 09:25:58.338454 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 09:25:58.338912 27033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } has no permanent_uuid. Determining permanent_uuid...
I20260620 09:25:58.339283 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.339157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39665 (local address 127.0.0.1:47604)
0620 09:25:58.339216 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.339252 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 09:25:58.339661 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.339562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39665 (local address 127.0.0.1:47606)
0620 09:25:58.339611 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.339642 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 09:25:58.339751 27033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.60:39665: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39665: connect: Connection refused (error 111)
I20260620 09:25:58.370913 27033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } attempt: 1
I20260620 09:25:58.371403 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.371238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39665 (local address 127.0.0.1:47608)
0620 09:25:58.371321 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.371368 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 09:25:58.371778 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.371671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39665 (local address 127.0.0.1:47614)
0620 09:25:58.371718 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.371745 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 09:25:58.371873 27033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.60:39665: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39665: connect: Connection refused (error 111)
W20260620 09:25:58.373234 27115 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 09:25:58.373461 27115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 09:25:58.373507 27115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 09:25:58.373539 27115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 09:25:58.375115 27115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 09:25:58.375156 27115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 09:25:58.375170 27115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 09:25:58.375183 27115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 09:25:58.375195 27115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 09:25:58.376592 27115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.59.20:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.59.62:37875,127.4.59.61:32903,127.4.59.60:39665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/krb5kdc/kudu_127.4.59.60.keytab
--principal=kudu/127.4.59.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.59.60:39665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.59.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 08:22:15 UTC on ccycloud.quasar-qyrqrz.root.comops.site
build id 4457055
I20260620 09:25:58.376782 27115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 09:25:58.376947 27115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 09:25:58.381487 27120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:58.381562 27121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 09:25:58.381685 27123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 09:25:58.381703 27115 server_base.cc:1034] running on GCE node
I20260620 09:25:58.381937 27115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 09:25:58.382228 27115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 09:25:58.383396 27115 hybrid_clock.cc:648] HybridClock initialized: now 1781947558383370 us; error 35 us; skew 500 ppm
Jun 20 09:25:58 dist-test-slave-414w krb5kdc[26952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947558, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 09:25:58.385787 27115 init.cc:377] Logged in from keytab as kudu/127.4.59.60@KRBTEST.COM (short username kudu)
I20260620 09:25:58.386229 27115 webserver.cc:460] Webserver started at http://127.4.59.60:43617/ using document root <none> and password file <none>
I20260620 09:25:58.386436 27115 fs_manager.cc:359] Metadata directory not provided
I20260620 09:25:58.386480 27115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 09:25:58.386631 27115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 09:25:58.387634 27115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/data/instance:
uuid: "f827b2adbecd4eaebbeb54b3b9976647"
format_stamp: "Formatted at 2026-06-20 09:25:58 on dist-test-slave-414w"
server_key: "af8a1671021cab646269502cd30d9f8b"
server_key_iv: "7ce5b522c3d11ca7da16c8464ada92b6"
server_key_version: "encryptionkey@0"
I20260620 09:25:58.387981 27115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/wal/instance:
uuid: "f827b2adbecd4eaebbeb54b3b9976647"
format_stamp: "Formatted at 2026-06-20 09:25:58 on dist-test-slave-414w"
server_key: "af8a1671021cab646269502cd30d9f8b"
server_key_iv: "7ce5b522c3d11ca7da16c8464ada92b6"
server_key_version: "encryptionkey@0"
I20260620 09:25:58.389273 27115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 09:25:58.389977 27130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 09:25:58.390147 27115 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 09:25:58.390228 27115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/data,/tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/wal
uuid: "f827b2adbecd4eaebbeb54b3b9976647"
format_stamp: "Formatted at 2026-06-20 09:25:58 on dist-test-slave-414w"
server_key: "af8a1671021cab646269502cd30d9f8b"
server_key_iv: "7ce5b522c3d11ca7da16c8464ada92b6"
server_key_version: "encryptionkey@0"
I20260620 09:25:58.390285 27115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 09:25:58.395790 27108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } attempt: 2
I20260620 09:25:58.396108 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.395995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39665 (local address 127.0.0.1:47616)
0620 09:25:58.396049 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.396080 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 09:25:58.396405 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.396347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39665 (local address 127.0.0.1:47624)
0620 09:25:58.396369 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.396392 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260620 09:25:58.396495 27108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.59.60:39665: Network error: Client connection negotiation failed: client connection to 127.4.59.60:39665: connect: Connection refused (error 111)
I20260620 09:25:58.408511 27115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 09:25:58.410295 27115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 09:25:58.410439 27115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 09:25:58.414086 27115 rpc_server.cc:307] RPC server started. Bound to: 127.4.59.60:39665
I20260620 09:25:58.414119 27182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.59.60:39665 every 8 connection(s)
I20260620 09:25:58.414448 27115 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/data/info.pb
I20260620 09:25:58.415392 27183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 09:25:58.416169 4332 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Fx1M2/build/release/bin/kudu as pid 27115
I20260620 09:25:58.416290 4332 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Fx1M2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781947262065055-4332-0/minicluster-data/master-2/wal/instance
I20260620 09:25:58.416488 4332 external_mini_cluster.cc:1428] Setting key 85a03c5b2836814e48437a06f927b5a1
I20260620 09:25:58.417338 27183 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 09:25:58 dist-test-slave-414w krb5kdc[26952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947558, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.60@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
Jun 20 09:25:58 dist-test-slave-414w krb5kdc[26952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.60@KRBTEST.COM
I20260620 09:25:58.426460 27036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.418274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46642 (local address 127.4.59.62:37875)
0620 09:25:58.418322 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.418324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.418464 (+ 140us) server_negotiation.cc:407] Connection header received
0620 09:25:58.418549 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.418550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.418561 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.418628 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.419046 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.419747 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.420817 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.421000 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.421107 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.423278 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:58.423279 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:58.423279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:58.423287 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:58.424915 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.425411 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.425412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.425412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.425462 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.425726 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.425727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.425728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.425862 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:58.425943 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:58.426316 (+ 373us) server_negotiation.cc:299] Negotiation successful
0620 09:25:58.426378 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 09:25:58.426658 27187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.418175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.62:37875 (local address 127.0.0.1:46642)
0620 09:25:58.418351 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.418394 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:58.418471 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:58.418651 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:58.418654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:58.418873 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:58.418958 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.418962 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.419903 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.419904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.420719 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.420721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.421172 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.421173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.421242 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.421829 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:58.421837 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:58.423191 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:58.424990 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.424994 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.424996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.425341 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.425528 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.425529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.425529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.425657 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.425983 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:58.425987 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:58.426145 (+ 158us) client_negotiation.cc:769] Sending connection context
0620 09:25:58.426278 (+ 133us) client_negotiation.cc:240] Negotiation successful
0620 09:25:58.426432 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":104,"threads_started":1}
I20260620 09:25:58.426851 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.416971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47632 (local address 127.4.59.60:39665)
0620 09:25:58.417176 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.417185 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.417216 (+ 31us) server_negotiation.cc:407] Connection header received
0620 09:25:58.417625 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.417632 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.417674 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.417818 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.418582 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.419843 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.420578 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.420975 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.421049 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.423776 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:58.423783 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:58.423783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:58.423789 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:58.425574 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.425988 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.425991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.425991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.426057 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.426251 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.426252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.426252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.426402 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:58.426528 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:58.426647 (+ 119us) server_negotiation.cc:299] Negotiation successful
0620 09:25:58.426732 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260620 09:25:58.427336 27183 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 09:25:58 dist-test-slave-414w krb5kdc[26952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947558, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.60@KRBTEST.COM for kudu/127.4.59.61@KRBTEST.COM
Jun 20 09:25:58 dist-test-slave-414w krb5kdc[26952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.62@KRBTEST.COM
I20260620 09:25:58.434512 27187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.427575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.61:32903 (local address 127.0.0.1:50158)
0620 09:25:58.427623 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.427633 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:58.427675 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:58.427830 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:58.427832 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:58.427963 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:58.427989 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.427990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.428631 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.428632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.429097 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.429098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.429540 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.429541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.429597 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.429968 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:58.429973 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:58.431188 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:58.433212 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.433213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.433214 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.433393 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.433830 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.433830 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.433831 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.433879 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.434182 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:58.434183 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:58.434218 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 09:25:58.434311 (+ 93us) client_negotiation.cc:240] Negotiation successful
0620 09:25:58.434401 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 09:25:58.434514 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.427633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50158 (local address 127.4.59.61:32903)
0620 09:25:58.427678 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.427679 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.427695 (+ 16us) server_negotiation.cc:407] Connection header received
0620 09:25:58.427742 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.427743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.427750 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.427809 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.428067 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.428527 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.429235 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.429455 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.429540 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.431275 (+ 1735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:58.431276 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:58.431277 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:58.431283 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:58.433100 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.433470 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.433472 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.433472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.433531 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.433963 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.433964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.433964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.434116 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:58.434183 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:58.434345 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 09:25:58.434423 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:25:58.434547 27036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.427655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46654 (local address 127.4.59.62:37875)
0620 09:25:58.427691 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.427693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.427756 (+ 63us) server_negotiation.cc:407] Connection header received
0620 09:25:58.427830 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.427832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.427840 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.427894 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.428335 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.428768 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.429510 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.429722 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.429790 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.431907 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:58.431908 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:58.431909 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:58.431916 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:58.433539 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.433820 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.433821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.433821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.433868 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.434063 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.434064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.434064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.434202 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:58.434352 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:58.434395 (+ 43us) server_negotiation.cc:299] Negotiation successful
0620 09:25:58.434463 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:25:58.435060 27183 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 09:25:58 dist-test-slave-414w krb5kdc[26952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947558, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.60@KRBTEST.COM for kudu/127.4.59.60@KRBTEST.COM
Jun 20 09:25:58 dist-test-slave-414w krb5kdc[26952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.59.61@KRBTEST.COM
I20260620 09:25:58.442533 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.435351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47634 (local address 127.4.59.60:39665)
0620 09:25:58.435390 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.435391 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.435430 (+ 39us) server_negotiation.cc:407] Connection header received
0620 09:25:58.435526 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.435527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.435538 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.435600 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.435877 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.436548 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.437416 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.437652 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.437743 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.439626 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:58.439627 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:58.439628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:58.439633 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:58.441393 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.441700 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.441701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.441701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.441765 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.441948 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.441949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.441949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.442085 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:58.442145 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:58.442296 (+ 151us) server_negotiation.cc:299] Negotiation successful
0620 09:25:58.442379 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 09:25:58.442543 27187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.435320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39665 (local address 127.0.0.1:47634)
0620 09:25:58.435390 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.435400 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:58.435444 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:58.435611 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:58.435612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:58.435762 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:58.435797 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.435799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.436622 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.436623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.437337 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.437339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.437742 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.437743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.437780 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.438158 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:58.438162 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:58.439501 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:58.441468 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.441469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.441470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.441626 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.441835 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.441836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.441836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.441886 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.442151 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:58.442151 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:58.442197 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 09:25:58.442281 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 09:25:58.442421 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":47}
I20260620 09:25:58.443306 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.435475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50168 (local address 127.4.59.61:32903)
0620 09:25:58.435505 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.435507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.435514 (+ 7us) server_negotiation.cc:407] Connection header received
0620 09:25:58.435554 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.435555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.435564 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.435613 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.436123 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.436763 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.437615 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.437889 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.437946 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.440477 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:58.440478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:58.440479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:58.440486 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:58.442021 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.442400 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.442401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.442401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.442453 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.442686 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.442687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.442687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.442818 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:58.442979 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:58.443184 (+ 205us) server_negotiation.cc:299] Negotiation successful
0620 09:25:58.443241 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 09:25:58.444384 27183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647: Bootstrap starting.
I20260620 09:25:58.444749 4332 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 09:25:58.445497 27183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:58.445820 27183 log.cc:824] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:58.446640 27183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647: No bootstrap required, opened a new log
I20260620 09:25:58.446031 27007 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "80f6ed82ef1048b88bb83d631ae93fc4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46654
I20260620 09:25:58.447003 27033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } attempt: 2
I20260620 09:25:58.448614 27183 raft_consensus.cc:348] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } }
I20260620 09:25:58.448760 27183 raft_consensus.cc:729] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f827b2adbecd4eaebbeb54b3b9976647, State: Initialized, Role: FOLLOWER
I20260620 09:25:58.448849 27183 consensus_queue.cc:260] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } }
I20260620 09:25:58.449118 27189 sys_catalog.cc:455] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } } }
I20260620 09:25:58.449276 27189 sys_catalog.cc:458] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:25:58.449208 27183 sys_catalog.cc:564] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647 [sys.catalog]: configured and running, proceeding with master startup.
Jun 20 09:25:58 dist-test-slave-414w krb5kdc[26952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947558, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.62@KRBTEST.COM for kudu/127.4.59.60@KRBTEST.COM
W20260620 09:25:58.451383 27200 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:25:58.451433 27200 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:25:58.454300 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.447290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39665 (local address 127.0.0.1:47650)
0620 09:25:58.447336 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.447345 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:58.447383 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:58.447532 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:58.447534 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:58.447674 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:58.447708 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.447709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.448304 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.448306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.448800 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.448801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.449322 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.449323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.449395 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.449787 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:58.449792 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:58.451253 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:58.453051 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.453052 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.453053 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.453324 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.453564 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.453565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.453566 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.453646 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.453982 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:58.453983 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:58.454034 (+ 51us) client_negotiation.cc:769] Sending connection context
0620 09:25:58.454111 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 09:25:58.454206 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 09:25:58.454310 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.447345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47650 (local address 127.4.59.60:39665)
0620 09:25:58.447380 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.447382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.447390 (+ 8us) server_negotiation.cc:407] Connection header received
0620 09:25:58.447442 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.447443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.447451 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.447504 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.447792 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.448221 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.448913 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.449221 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.449296 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.451336 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:58.451337 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:58.451338 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:58.451344 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:58.452959 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.453419 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.453421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.453421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.453489 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.453717 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.453718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.453718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.453887 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:58.453954 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:58.454146 (+ 192us) server_negotiation.cc:299] Negotiation successful
0620 09:25:58.454222 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:25:58.452369 27115 master_runner.cc:186] Error getting master registration for 127.4.59.62:37875: OK, instance_id { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" instance_seqno: 1781947558106251 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 09:25:58.455209 27115 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 09:25:58.455806 27033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4: Bootstrap starting.
I20260620 09:25:58.456501 27033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:58.456820 27033 log.cc:824] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:58.457671 27033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4: No bootstrap required, opened a new log
I20260620 09:25:58.459136 27033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } }
I20260620 09:25:58.459280 27033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80f6ed82ef1048b88bb83d631ae93fc4, State: Initialized, Role: FOLLOWER
I20260620 09:25:58.459358 27033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } }
I20260620 09:25:58.459648 27033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:58.459738 27007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 09:25:58.459838 27007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:58.460994 27007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } }
I20260620 09:25:58.461553 27203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } } }
I20260620 09:25:58.461659 27203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:25:58.461604 27157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f827b2adbecd4eaebbeb54b3b9976647"
I20260620 09:25:58.461724 27157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647 [term 0 FOLLOWER]: Advancing to term 1
W20260620 09:25:58.462284 27214 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:25:58.462354 27214 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:25:58.461325 27007 leader_election.cc:288] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [CANDIDATE]: Term 1 election: Requested vote from peers 0c5a552c9add4fcf9ed4e681e8aace72 (127.4.59.61:32903), f827b2adbecd4eaebbeb54b3b9976647 (127.4.59.60:39665)
I20260620 09:25:58.463081 27157 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80f6ed82ef1048b88bb83d631ae93fc4 in term 1.
I20260620 09:25:58.462513 26965 master_runner.cc:186] Error getting master registration for 127.4.59.61:32903: OK, instance_id { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" instance_seqno: 1781947558254727 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 09:25:58.466163 26965 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 09:25:58.466638 26982 leader_election.cc:302] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80f6ed82ef1048b88bb83d631ae93fc4, f827b2adbecd4eaebbeb54b3b9976647; no voters:
I20260620 09:25:58.466894 27203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 09:25:58.467093 27203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [term 1 LEADER]: Becoming Leader. State: Replica: 80f6ed82ef1048b88bb83d631ae93fc4, State: Running, Role: LEADER
I20260620 09:25:58.467197 27203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } }
I20260620 09:25:58.467612 27203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80f6ed82ef1048b88bb83d631ae93fc4. Latest consensus state: current_term: 1 leader_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } } }
I20260620 09:25:58.467765 27203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:58.467984 27224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 09:25:58.468174 27224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 09:25:58.470393 27157 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647 [term 1 FOLLOWER]: Refusing update from remote peer 80f6ed82ef1048b88bb83d631ae93fc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:25:58.470710 27203 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 09:25:58.472056 27189 sys_catalog.cc:455] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80f6ed82ef1048b88bb83d631ae93fc4. Latest consensus state: current_term: 1 leader_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } } }
I20260620 09:25:58.472144 27189 sys_catalog.cc:458] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:25:58.474855 27225 mvcc.cc:205] Tried to move back new op lower bound from 7298857199492014080 to 7298857199482535936. Current Snapshot: MvccSnapshot[applied={T|T < 7298857199492014080}]
I20260620 09:25:58.474984 27036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.463636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46668 (local address 127.4.59.62:37875)
0620 09:25:58.463675 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.463678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.464428 (+ 750us) server_negotiation.cc:407] Connection header received
0620 09:25:58.464581 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.464582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.464604 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.464635 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.465879 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.466601 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.468335 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.468638 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.468679 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.471247 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:58.471260 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:58.471261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:58.471267 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:58.473072 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.473828 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.473829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.473829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.473887 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.474209 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.474210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.474210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.474346 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:58.474448 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:58.474805 (+ 357us) server_negotiation.cc:299] Negotiation successful
0620 09:25:58.474888 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:25:58.475126 27203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } } }
I20260620 09:25:58.475200 27203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:58.475617 27189 sys_catalog.cc:455] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } } }
I20260620 09:25:58.475718 27189 sys_catalog.cc:458] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:25:58.475899 27226 mvcc.cc:205] Tried to move back new op lower bound from 7298857199492014080 to 7298857199482535936. Current Snapshot: MvccSnapshot[applied={T|T < 7298857199492014080}]
I20260620 09:25:58.481251 27224 catalog_manager.cc:1347] Generated new cluster ID: bac480b0d82b4f9fbbab1be36941663b
I20260620 09:25:58.481302 27224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 09:25:58.483050 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.464039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50174 (local address 127.4.59.61:32903)
0620 09:25:58.464079 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.464081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.473272 (+ 9191us) server_negotiation.cc:407] Connection header received
0620 09:25:58.473344 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.473346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.473359 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.473411 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.474465 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.475181 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.476701 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.477017 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.477062 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.479307 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:58.479309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:58.479309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:58.479316 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:58.481432 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.482009 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.482010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.482010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.482075 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.482357 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.482358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.482358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.482492 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:58.482575 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:58.482873 (+ 298us) server_negotiation.cc:299] Negotiation successful
0620 09:25:58.482951 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 09:25:58.487663 27108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } attempt: 3
Jun 20 09:25:58 dist-test-slave-414w krb5kdc[26952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781947558, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.59.61@KRBTEST.COM for kudu/127.4.59.60@KRBTEST.COM
I20260620 09:25:58.497294 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.473507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47656 (local address 127.4.59.60:39665)
0620 09:25:58.473550 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.473552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.475911 (+ 2359us) server_negotiation.cc:407] Connection header received
0620 09:25:58.476054 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.476056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.476066 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.476095 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.477067 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.477845 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.479237 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.479539 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.479587 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.493204 (+ 13617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:58.493205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:58.493206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:58.493215 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:58.494880 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.495742 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.495744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.495744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.495811 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.496302 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.496303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.496303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.496446 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:58.496519 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:58.497068 (+ 549us) server_negotiation.cc:299] Negotiation successful
0620 09:25:58.497194 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 09:25:58.499888 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.487962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.59.60:39665 (local address 127.0.0.1:47666)
0620 09:25:58.488060 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0620 09:25:58.488070 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 09:25:58.488107 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 09:25:58.488555 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 09:25:58.488556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 09:25:58.488728 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 09:25:58.488774 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.488776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.490289 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.490290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.491649 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 09:25:58.491652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.492512 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.492513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 09:25:58.492571 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.493109 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 09:25:58.493114 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 09:25:58.495198 (+ 2084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 09:25:58.498475 (+ 3277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.498476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.498477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.498692 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.499050 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 09:25:58.499051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 09:25:58.499051 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 09:25:58.499119 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 09:25:58.499584 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 09:25:58.499585 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 09:25:58.499640 (+ 55us) client_negotiation.cc:769] Sending connection context
0620 09:25:58.499686 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 09:25:58.499789 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260620 09:25:58.500536 27232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:58.488186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47666 (local address 127.4.59.60:39665)
0620 09:25:58.488345 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.488348 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.488378 (+ 30us) server_negotiation.cc:407] Connection header received
0620 09:25:58.488399 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.488400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.488443 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.488855 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.488923 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.490140 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.491821 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:58.492394 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:58.495292 (+ 2898us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:58.495364 (+ 72us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 09:25:58.495365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 09:25:58.495366 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 09:25:58.495391 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 09:25:58.498339 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.498805 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.498805 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.498806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.498934 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 09:25:58.499205 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 09:25:58.499206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 09:25:58.499206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 09:25:58.499466 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 09:25:58.500113 (+ 647us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:58.500220 (+ 107us) server_negotiation.cc:299] Negotiation successful
0620 09:25:58.500346 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260620 09:25:58.502096 27108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72: Bootstrap starting.
I20260620 09:25:58.502863 27108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72: Neither blocks nor log segments found. Creating new log.
I20260620 09:25:58.503167 27108 log.cc:824] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72: Log is configured to *not* fsync() on all Append() calls
I20260620 09:25:58.504097 27108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72: No bootstrap required, opened a new log
I20260620 09:25:58.506126 27108 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } }
I20260620 09:25:58.506265 27108 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c5a552c9add4fcf9ed4e681e8aace72, State: Initialized, Role: FOLLOWER
I20260620 09:25:58.506351 27108 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } }
I20260620 09:25:58.506671 27108 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 09:25:58.506758 27082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0c5a552c9add4fcf9ed4e681e8aace72"
I20260620 09:25:58.506876 27082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72 [term 0 FOLLOWER]: Advancing to term 1
I20260620 09:25:58.507706 27082 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80f6ed82ef1048b88bb83d631ae93fc4 in term 1.
I20260620 09:25:58.508198 27081 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72 [term 1 FOLLOWER]: Refusing update from remote peer 80f6ed82ef1048b88bb83d631ae93fc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 09:25:58.508512 27230 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 09:25:58.509302 27245 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 09:25:58.509358 27245 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 09:25:58.511445 27234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } } }
I20260620 09:25:58.511538 27234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:25:58.511628 27234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80f6ed82ef1048b88bb83d631ae93fc4. Latest consensus state: current_term: 1 leader_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } } }
I20260620 09:25:58.511682 27234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:25:58.511718 27234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } } }
I20260620 09:25:58.511768 27234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 09:25:58.517128 27248 mvcc.cc:205] Tried to move back new op lower bound from 7298857199492014080 to 7298857199482535936. Current Snapshot: MvccSnapshot[applied={T|T < 7298857199492014080}]
I20260620 09:25:58.518275 27230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f6ed82ef1048b88bb83d631ae93fc4" member_type: VOTER last_known_addr { host: "127.4.59.62" port: 37875 } } peers { permanent_uuid: "0c5a552c9add4fcf9ed4e681e8aace72" member_type: VOTER last_known_addr { host: "127.4.59.61" port: 32903 } } peers { permanent_uuid: "f827b2adbecd4eaebbeb54b3b9976647" member_type: VOTER last_known_addr { host: "127.4.59.60" port: 39665 } } }
I20260620 09:25:58.518373 27230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4 [sys.catalog]: This master's current role is: LEADER
I20260620 09:25:58.520237 27224 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 09:25:58.520948 27224 catalog_manager.cc:1504] Loading token signing keys...
I20260620 09:25:58.533898 27224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80f6ed82ef1048b88bb83d631ae93fc4: Generated new TSK 0
I20260620 09:25:58.534106 27224 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 09:25:58.563431 27036 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:58.561829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46678 (local address 127.4.59.62:37875)
0620 09:25:58.561875 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.561877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.562763 (+ 886us) server_negotiation.cc:407] Connection header received
0620 09:25:58.562872 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.562873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.562884 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.562915 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.563361 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46678: BlockingRecv error: recv got EOF from 127.0.0.1:46678 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 09:25:58.564483 27112 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:58.562179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50190 (local address 127.4.59.61:32903)
0620 09:25:58.562220 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.562222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.563665 (+ 1443us) server_negotiation.cc:407] Connection header received
0620 09:25:58.563924 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.563925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.563936 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.563966 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.564410 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50190: BlockingRecv error: recv got EOF from 127.0.0.1:50190 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 09:25:58.564937 27232 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:58.562507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47680 (local address 127.4.59.60:39665)
0620 09:25:58.562572 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.562574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.563787 (+ 1213us) server_negotiation.cc:407] Connection header received
0620 09:25:58.564029 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.564031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.564040 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.564069 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.564875 (+ 806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47680: BlockingRecv error: recv got EOF from 127.0.0.1:47680 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260620 09:25:58.567251 27112 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:58.566549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50204 (local address 127.4.59.61:32903)
0620 09:25:58.566594 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.566595 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.566768 (+ 173us) server_negotiation.cc:407] Connection header received
0620 09:25:58.566867 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.566868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.566883 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.566912 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.567210 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50204: BlockingRecv error: recv got EOF from 127.0.0.1:50204 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260620 09:25:58.570214 27112 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:58.569470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50220 (local address 127.4.59.61:32903)
0620 09:25:58.569500 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.569501 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.569751 (+ 250us) server_negotiation.cc:407] Connection header received
0620 09:25:58.569844 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.569845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.569855 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.569882 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.570174 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50220: BlockingRecv error: recv got EOF from 127.0.0.1:50220 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260620 09:25:58.573993 27112 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:58.573246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50226 (local address 127.4.59.61:32903)
0620 09:25:58.573278 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.573280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.573510 (+ 230us) server_negotiation.cc:407] Connection header received
0620 09:25:58.573622 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.573623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.573633 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.573662 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.573956 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50226: BlockingRecv error: recv got EOF from 127.0.0.1:50226 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260620 09:25:58.579680 27112 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:58.578931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50242 (local address 127.4.59.61:32903)
0620 09:25:58.578963 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.578965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.579195 (+ 230us) server_negotiation.cc:407] Connection header received
0620 09:25:58.579288 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.579289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.579299 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.579327 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.579642 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50242: BlockingRecv error: recv got EOF from 127.0.0.1:50242 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260620 09:25:58.589587 27112 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:58.588688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50256 (local address 127.4.59.61:32903)
0620 09:25:58.588734 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.588736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.588964 (+ 228us) server_negotiation.cc:407] Connection header received
0620 09:25:58.589059 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.589060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.589070 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.589099 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.589520 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50256: BlockingRecv error: recv got EOF from 127.0.0.1:50256 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260620 09:25:58.607765 27112 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:58.606785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50272 (local address 127.4.59.61:32903)
0620 09:25:58.606826 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.606829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.607098 (+ 269us) server_negotiation.cc:407] Connection header received
0620 09:25:58.607246 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.607248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.607283 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.607316 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.607708 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50272: BlockingRecv error: recv got EOF from 127.0.0.1:50272 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260620 09:25:58.641942 27112 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:58.641206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50286 (local address 127.4.59.61:32903)
0620 09:25:58.641261 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.641264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.641427 (+ 163us) server_negotiation.cc:407] Connection header received
0620 09:25:58.641496 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.641497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.641508 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.641565 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.641884 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50286: BlockingRecv error: recv got EOF from 127.0.0.1:50286 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 09:25:58.707705 27112 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:58.707044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50300 (local address 127.4.59.61:32903)
0620 09:25:58.707158 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.707161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.707174 (+ 13us) server_negotiation.cc:407] Connection header received
0620 09:25:58.707247 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.707248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.707258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.707307 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.707626 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50300: BlockingRecv error: recv got EOF from 127.0.0.1:50300 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260620 09:25:58.837705 27112 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:58.836892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50310 (local address 127.4.59.61:32903)
0620 09:25:58.836989 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:58.836991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:58.837085 (+ 94us) server_negotiation.cc:407] Connection header received
0620 09:25:58.837211 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:58.837213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:58.837224 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:58.837293 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:58.837636 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50310: BlockingRecv error: recv got EOF from 127.0.0.1:50310 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260620 09:25:59.095815 27112 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 09:25:59.095145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50318 (local address 127.4.59.61:32903)
0620 09:25:59.095218 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:59.095220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:59.095231 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:25:59.095295 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:59.095296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:59.095309 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:59.095381 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 09:25:59.095733 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50318: BlockingRecv error: recv got EOF from 127.0.0.1:50318 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 09:25:59.451897 27200 catalog_manager.cc:1259] Loaded cluster ID: bac480b0d82b4f9fbbab1be36941663b
I20260620 09:25:59.451989 27200 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647: loading cluster ID for follower catalog manager: success
I20260620 09:25:59.452764 27200 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647: acquiring CA information for follower catalog manager: success
I20260620 09:25:59.453007 27200 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f827b2adbecd4eaebbeb54b3b9976647: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 09:25:59.509773 27245 catalog_manager.cc:1259] Loaded cluster ID: bac480b0d82b4f9fbbab1be36941663b
I20260620 09:25:59.509850 27245 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72: loading cluster ID for follower catalog manager: success
I20260620 09:25:59.510664 27245 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72: acquiring CA information for follower catalog manager: success
I20260620 09:25:59.510967 27245 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0c5a552c9add4fcf9ed4e681e8aace72: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 09:25:59.611272 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:59.608857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35542 (local address 127.4.59.61:32903)
0620 09:25:59.608959 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:59.608961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:59.608971 (+ 10us) server_negotiation.cc:407] Connection header received
0620 09:25:59.609044 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:59.609045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:59.609090 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:59.609214 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:25:59.609354 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:59.609850 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:59.610578 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:59.610760 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:59.610810 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:59.610946 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:25:59.611051 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:25:59.611109 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:59.611145 (+ 36us) server_negotiation.cc:299] Negotiation successful
0620 09:25:59.611162 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73,"thread_start_us":48,"threads_started":1}
I20260620 09:25:59.615298 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:59.613123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34354 (local address 127.4.59.60:39665)
0620 09:25:59.613253 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:59.613254 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:59.613265 (+ 11us) server_negotiation.cc:407] Connection header received
0620 09:25:59.613300 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:59.613300 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:59.613332 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:59.613397 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:25:59.613463 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:59.613942 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:59.614616 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:59.614813 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:59.614868 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:59.615034 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:25:59.615125 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:25:59.615181 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:59.615222 (+ 41us) server_negotiation.cc:299] Negotiation successful
0620 09:25:59.615237 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260620 09:25:59.619855 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:59.616888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35042 (local address 127.4.59.62:37875)
0620 09:25:59.616984 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:59.616986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:59.617080 (+ 94us) server_negotiation.cc:407] Connection header received
0620 09:25:59.617181 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:59.617182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:59.617231 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:59.617307 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:25:59.617429 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:59.617977 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:59.618926 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:59.619255 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:59.619319 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:59.619476 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:25:59.619582 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:25:59.619639 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:59.619745 (+ 106us) server_negotiation.cc:299] Negotiation successful
0620 09:25:59.619766 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":45,"threads_started":1}
I20260620 09:25:59.620160 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:59.616952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34366 (local address 127.4.59.60:39665)
0620 09:25:59.617024 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:59.617027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:59.617171 (+ 144us) server_negotiation.cc:407] Connection header received
0620 09:25:59.617269 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:59.617270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:59.617280 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:59.617325 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:25:59.617491 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:59.618252 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:59.619128 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:59.619437 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:59.619501 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:59.619617 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:25:59.619644 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:25:59.619719 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:59.620043 (+ 324us) server_negotiation.cc:299] Negotiation successful
0620 09:25:59.620070 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 09:25:59.620398 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 09:25:59.617060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35548 (local address 127.4.59.61:32903)
0620 09:25:59.617095 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 09:25:59.617097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 09:25:59.617313 (+ 216us) server_negotiation.cc:407] Connection header received
0620 09:25:59.617392 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 09:25:59.617394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 09:25:59.617404 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 09:25:59.617457 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 09:25:59.617570 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:59.618270 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:59.619257 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 09:25:59.619539 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 09:25:59.619608 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 09:25:59.620124 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 09:25:59.620158 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 09:25:59.620187 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0620 09:25:59.620308 (+ 121us) server_negotiation.cc:299] Negotiation successful
0620 09:25:59.620330 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 09:25:59.626519 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 26965
I20260620 09:25:59.632133 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 27040
I20260620 09:25:59.636858 4332 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Fx1M2/build/release/bin/kudu with pid 27115
2026-06-20T09:25:59Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2804 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5612 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (297578 ms total)
[ PASSED ] 54 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
2 FAILED TESTS